Thursday, 24 October 2013

What the micro-framework?

I'm impressed by flask. I'm not a great pythonista so you will have to make up your own mind, but when I found Flask Bone my first thought was, . o O ( What is the perl equivalent to fbone? )

I've spent many years hacking Notice together, but it is far from web 2.0, (which is probably why it gets used for internal sites more than external sites, but with HTML5 Boilerplate and bootstrap it scrubs up nicely.)

So how did I find fbone? I was considering porting Notice, (which is mostly perl + jQuery ) to pyNotice, (a proposed python + jQuery.) I didn't want to re-invent the wheel, (though sometimes that can be a good thing), so I did a quick poke to see what was already out there.

Not all websites are created equal, but here are a few things that almost every site, (larger than a blog) need:
  • Login
    • Sign-up
    • Remember me
    • Forgotten password
  • Logout
  • Sessions
    • Client side data
    • Server side data
  • User data
  • File upload
  • Ajax of some sort to improve user experience
  • CSS control, (let the use change this as then need
All of this and we have not even got to the actual functionality of the site. Notice, (which is, at this time, one big perl module) splits its functionality into groups, (that are also called modules, but are not individual modules... yet.)

If you are a web developer that used perl or python, (keep your gems and php to yourself please) what do you use to jumpstart a website? Do you have a stock core code that you use? Is it open-source? Would Richard Stallman or Rev K rage at your code or rate it?

How far does this scale horizontally and vertically? Should we all be moving to happstack?

Wednesday, 16 October 2013

Haskell Data.Map.Lazy

cabal install containers # import Data.Map.Lazy

I have been enjoying the Haskell from Scratch series from Jekor over on youtube.

After checking out his example code from github I have been able to compile along with the music... until ep07

./redo redo 1: redo-ifchange: not found

    Could not find module `Data.Map.Lazy'
Redo script exited with non-zero exit code: 1

"Ah ha!" I thought to myself

cabal install Data.Map.Lazy

Nope. Off to Hoogle I go and after a while I worked out that what I needed was:

cabal install containers

but it was not Hoogle, but Google that clued me in.

cabal list|grep -i Map|grep -i Lazy

didn't help. Now that I know the answer it is quite easy to find:

hoogle --info Data.Map.Lazy|grep 'From package'|awk '{print $NF}'

and even create a general Haskell module installer:

cabal install $(hoogle --info $1 | grep 'From package'|awk '{print $NF}')

Sunday, 29 September 2013

An appreciating appreciation of appreciating appreciation

"Cream always floats to the top" is a saying that I learn on a farm. The idea is that if there is something of quality it will eventually show up. Like many sayings, "Feed a cold, starve a fever" [0] it is easy to pick holes and even easier to miss the idea completely.

When I found out about subbable one comment mentioned and I thought of flattr. My first reaction was, "do we need another?" Then I remembered that the best things evolve, (because how could I chose?) and for that to happen there need﻿ to be contestants. Just as KickStarter seems to have won the CrwdFund crown, it will be interesting to see which micropatronage site comes out on top. we have to keep track.

This morning C.G.P. Grey became the second passenger upon the micro-paetron train.
(A very worthy addition, in my opinion.)
[0] This is advice to calm a new mother that is panicking about their ill child. It is NOT medical advice for how to treat those with a fever. 

Friday, 27 September 2013

Unix unix everywhere and not a drop to sync!

I use Linux. I'd love to say that I use Unix, but it has been many years since I used SunOS, (see what I did there) or BSD in anger, (I did it again, but different!)

Looking at the Linux family tree, it can leave you wondering, "am I using the right one?"

So here is my very opinionated, (soon to be out-of-date) guide.

Looking at some comparisons it is clear that Linux, (the kernel) has more outfits than Barbie and more gadgets than Bond, MacGyver and the A-Team combined.

So for maintenance I used Finnix, (this is what I have on a USB keyring).
For desktop/workstation/laptop/terminal/netbooks I use crunchbang, (with LXDE).
For my family members I use Ubuntu MintLinux, (Apple Mac of the Linux world.)
For my phone I (would) use Android, (oh Samsung, let me count the ways...)
For my servers I use Slackware and CentOS, (ok, I have some Debian servers, but don't tell anyone!)

I'd love to say that I use Gentoo or more specifically Sabayon, but I don't anymore.

Before you comment I should note that:

from 1998-2005 I used, (almost exclusively Slackware.) I've used SuSE, (don't know why I didn't like it.)
I've used Xubuntu, until I found #! (how do I add an exclamation mark to that without it being confusing?)

I've used ext2, ext3, ext4, ReiserFS, SquashFS.

I started out, (X11) using WM then tried blackbox, (too leet for me), and KDE, (too large for my needs.) Then I spent a few years using Gnome, (also large, but Moore's law meant that computers had caught up.) xfce, openbox, lxde are more my thing.

tar/make, pkgutils, RPM, APT, YUM, YaST.

I prefer SELinux to AppArmor and I've never use PaX.

So the GNU/Linux ecosystem is cluttered, but that is a natural thing. Each strain evolves from another, or bursts into life to fill a niche. Eventually more and more of them will die off leaving one or two at the pinnacle of each category:

  • art, audio, design, multimedia
  • computer lab
  • desktop, workstation
  • embedded
  • general, lightweight, portable, netbooks
  • live, auditing, maintenance, security
  • network, gateway
  • science
  • server
  • supercomputer
Now I imagine that the software world will match the Linux OS world until we end up with each non-embedded Linux computer, depending on what you need it to do, will simply be a matter of setting it to a particular node type, (or combining two or more.) I imagine that "server" would be incompatable with "desktop" or "multimedia" [0] but both multimedia and server would both require imagemagic. The question then becomes, how does the average user locate the best packages for their need. Rather than a [Start] button, there should be a [I want to:] button which gives various levels. If the user wants to draw, then there is everything from Tux Paint to GIMP and Inkscape. Identifing the users need with as little information as possible is one of the next tricks. With a webcam the computer can estimate the age of the person, and with the speed of typing it can estimate their skill level. (This becomes mute when commands are entered via verbal command.) Maybe this will be another situation where shibboleet might be needed. Just as long as the UNITY interface dies, (that thing is sloooooow).

[0] Desktop requires GUI, server must NOT have a GUI. I did mention that this is my opinion.

Wednesday, 25 September 2013

New windows desktop, 2013?

So you have a new windows computer. You know, that like a child it is not ready for the scary Internet. So what do you need? This is my list for 2013. will install things for you:


Then add,

openoffice or libraoffice (they do the same, but have not yet merged back into one another.)
Spybot S&D
SpiderOak - because you NEED backup and you need it FIRST and secure.
GRAMPS - family can be complicated; you should keep track.

Other things that you might like, that I can recommend are:

 Chrome Browser
 Pidgin + OTR
 VLC (though sMplayer is also good)
 Winamp (deadbeef if you are on linux)
 Audacity (record or edit audio. AVIdemux for video).
 Java Java Runtime Environment (JRE) (to play on KGS)
 GIMP (why isn't this a verb yet? "This image has been gimped."
 AVG (though I usually use ClamWin or ClamAV.)
 Spybot 2
 7-Zip  - no really, we don't need anything else. (If Igor Pavlov is bored, he could add an optional par2 and SHA512 feature to 7-Zip so that archives and RARs could be verified. SSD bit-rot will become more of a worry in the future. 

Have a look at and

Friday, 13 September 2013

MUD Ew-Too


Econnect network with BBC micro and BBC master computers.







Wednesday, 11 September 2013

Give a little whistle

In one adaptation of the story of Pinocchio, the character of Jiminy Cricket is assigned as his conscience. I think that this is a marvellous parable for agnostics, but is only a good start.

I think that the story of a wooden boy with flees would have been better, (though why would a marionette have flees? More importantly, why flees?

Well, they are there but invisible. There can easily be, (and usually are) many of them. With the anthropomorphism of a children's story it is easy to suggest that one or more of the flees has some sort of dominance at any given time. As the story progresses that hierarchy may change. Possibly most importantly they can alter our protagonists behaviour despite being so small and seemingly irrelevant.

This would give the narrative the flexibility to discuss knowing that something is held to be legal wrong, (driving above the speed limit) but cultural-accepted-within-limits, (about 10% higher on the largest roads seems to be accepted by many people) leaving our flees, (mind) with knowing that it is wrong and acceptable at the same time.

( Maybe a wooden cat? Chat and her flees. )

So why else might we want flees? Well if Jiminy was squished it would be sad, but not so if one of our flees was harping on about some minor slight, "That person didn't use their car indicators, so I'm 0.0001 seconds later than I could have been." Then the lead character could work on their mental health by psychically squishing the flees that either talk drivel or are negatively distracting.

We can chose which mental dialogue to focus upon and which to ignore. I would like more examples of exercises where this is practised.

Saturday, 7 September 2013

Windows 7 adding admin user cmd command line

net user alexx passphrase /add
net localgroup administrators alexx /add
net user alexx /active:yes

but if your laptop is in french then the second command is:

net localgroup administrateurs alexx /add

est pour windows 7 en france.

Change "alexx" to the user that you want to add and passphrase to your own super-secret passphrase.

I found this when I was asked to repair a windows7 laptop that had only one admin user and trying to log in gave:

"Echec de l'ouverture de session par le service Service de profil utilisateur. Impossible de charger le profil d'utilisateur."

Interestingly I could reboot, (hammering F8) and boot into safe mode and then logging in as the admin worked fine.

This enabled me to add an additional admin user to repair the other one. had an interesting suggestion but in the end I just changed the name of the problem account, (which didn't match the underlying C:\Users\{username}) and created a new account with the old name. Then I copied all of the files over, and once the owner was happy I showed them where their "back up" was and how to install SpiderOak, (because like far too many people, they had no back up at all.)

 If they come back I'll move them to Mint Linux, (they aren't ready for the power of Crunchbang.)

[0] found on
Also could have been useful, had they forgotten their password. ( is another walk-through of the windows password reset.)

Monday, 19 August 2013

Linux species 2013

It is easy to forget that Linux is the kernel and not the whole operating system. That said, colloquially, "I'm using linux" seems to be an acceptable statement. If we look at the computer operating system world as organic, (which I would argue it very much is within the *NIX biome) we can identify which version of linux has evolved to best fill each niche.

I started out using Slackware, (yes it is a point of pride, but the choice had nothing to do with me.) For understanding Linux, just as reading the Lion book explained UNIX Slackware 7, (not sure what the later version are like) or LFS can be a bit of a cold shower of education. Probably very good for you, but no one would chose that if they knew the implications and had the choice, (after the fact, when you have the warm glow of knowledge it seems like _everyone_ should do the same.)

I'm writing this on crunchbang. Just like CFEditor's previous names crunchbang may, with time, come to be known as cb-linux. For the power user it seems to be perfect, but a power user is perfectly capable of working that our for themselves.

So who else is living the linux life? What distro should they be using?

It looks like the desktop/laptop world is moving away from Ubuntu to Mint, (based on my extensive four data point sample set.) The irony of "people giving up on Ubuntu due to Unity", (when you understand the original meaning of the word ubuntu) is funny.

The more I look at this it seems that the best communities gather around one person with one hand-waving idea, "I want it like this" and then others that agree vote with their (digital) feet.

You ask: What rot?

Linux: Linus, Slackware: Patrick, Crunchbang: Philip Newborough (though the crunchbang community itself seems more important now. Talking of which, Muppets: Jim Henson.)

Which leads me to the next question, (though I clearly have not finished the first.) People die, (no that's not a question) so what happens to the community that built up around them?  Well it either crumbles if it is an empire, or incorporates as a sudo-democracy, (or occasionally continues as a dynasty).

We can already see that with CentOS, (see I did go back). So why do I think that CentOS is _THE_ server OS? Because I don't want a homogeneous environment.

We can't all be Debian freaks ;-) Though looking at this trends graph it seems, at first glance that no one is going to be getting freaky by 2020. I think that, just as Google seem to be referring to the Linux under the Android under the hood less and less, Debian is silently under Ubuntu under Linux Mint. Those that we, (as humanity) need to work on Debian don't actually have to talk about it or search via google, anymore than you use a map to get to your office after the first day.

2013 Linux picks by Alexx Roche: {
Server: CentOS, Power-dekstop: Crunchbang, Parents: Linux Mint.

What other ecosystems are there that you live in, and which distro would you use?

Tuesday, 6 August 2013

Success! Someone died.

As with Love there are many types [0] of success.

"Success!" can be a deceleration of achieving or a eureka moment.

A company/society/cult/army/family are all demonstrations of the collective limited by the constraints of the system that they are operating within.

The problem with military success is that it is usually built upon the deaths of others, (which no longer seems glamorous or glorious.) Though this is not that interesting to me, ("Do not speak to me of great men for their success is built upon the bodies of thousands"), it is my introduction to the idea of the board-room, (and other office rooms) as war. If we accept corporations as devices to make money rather than conquering people or territory, (though these two would not be totally alien to some corporations), it does seem to look a lot like a type of war.

The interesting thing is merging research into ethics, (another of my interests): As a hypothesis, "The more successful individuals within a corporation are inherently less ethical" seems to be showing positive results from tests. [1] So where success is simply the massive accruement of money or wealth, once again it seems to be build upon those that you can stand upon.

It does not seem to matter if you are "waiting for dead men's shoes" or "counting your chickens before they hatch" positive success should be built upon the achievements of others and not to the detriment of others.

The problem here is syncronicity. The light-bulb was 'invented' by two people at the same time. This is not the only example of things that have been invented twice, but in each case the success of one inventor is always going to be to the detriment of another. If a position opens up within a company and more than one person applies for it, then one or more people are going to fail.

I'm not suggesting that failure is a bad thing, but in the instances where it is maliciously detrimental, (patents that are not being exercised [2] other than to collect revenue or prevent others) should not be supported.

[0] Philos Amos Bergos bOS moss

[1] Joseph P Cangemi and William Pfohl “Sociopaths in High Places” 2009

[2] If you hold the patent for the perfect light-bulb and do not produce it, simply so that you can sell more of your defective bulbs, then I see no reason why someone else should not be allowed to produced the patented object.

Higher level sysadmin

I've been interested in the genealogy and hierarchy of computer languages ever since I bumped into the conundrum, "If C is a higher level language than ALGOL or assembly, what is Perl?" Obviously there are many humorous responses to this question, but I want to answer it more philosophically.

The perfect language has yet to evolve, and like insect species, computer languages will evolve to eventually perfectly fit their niche.

Linux seems to have come out on top of the OS world, just as C seems to be the higher level language. My first problem is that higher and lower were fine when there were two levels, but now we have a whole mess of branches and directions. It is better to think of C as a rung on a ladder or the 2nd floor of a building. (Floor can be ambiguous, so I would suggest the word étage from French. Again you can argue that the first element of an array is a[0] or a[1] so lets call binary etage[1] and assembler as etage[10]; leaving C as etage[15].)

[So what is this all leading up to?]

Today I was asked by a friend, "I'm thinking of replacing my mail server with something more modern. I have been looking at ATMail, do you have any takes on it?" He went on, "I want a mail server where, ideally, caldav+cardav+webmail+imap are all synchronized."

Well that seems logical. So what did I say?

"In my (*nix) world, there is eventually a default solution to each protocol. 2013:
webmail: roundcube
SMTP: exim (though if I had to start again I might pick postfix)
IMAP: dovecot
caldav: I've written my own, but seems very interesting, but DAVIcal is _the_ one for now."

This leaves a large hole. "synchronised".  I wrote Notice to try and do just that. It was a clunky collection of casually connected CGI that was unmaintainable, (except by me) and 100% dependent upon mySQL.

Then I met DBIx::Class, (I was introduced by a huge(ly) popular guy, and when I looked under his frame I found CGI::Application.)

Notice version 3.0 was database agnostic. It does not, (for the most part) care if you MUST use postgreSQL or SQLite, (it could almost use CDB - the greatest of DJB's inventions, (actually the only one that I understand enough to fully support.))

This abstraction made me realise that we need emsg_service. emsg_service is the 100% agnostic electronic message platform. It does not care if you use exim or postfix. It does not mind if you want flat-file configuration or direct DB config, (so that dovecot has instant updates.) It does not mind if you want to replace exim with sendmail.

But wait,...

... there is more. emsg_service is also language agnostic. It can be implemented in perl or python or haskel. emsg_service is the idea of distributed performance with centralised control of digital communications. It should have an API, a command line interface and a GUI via HTTPs. It can scale from one server to perform all services to as many clusters for each function, as you need.

 It MUST implement its own pocket Certificate Authority, OR accept a certificate to enable it to hang off an existing company hierarchy. (So that internal security may be freely implemented.)

It must let users store data: IMAPs calDAV cardDAV PGP&SSH keys and transmit data SMTPs. (It should have a quota for each. Any quota MUST have at least per-user granularity.)

I would suggest that it link in with cfEngine and FAI/KS so that it can configure a bare metal machine with an OS just by knowing its MAC.

To that end it will need an asset management system to track the physical machines, their locations and the intended configuration. (And a tftp server.)

While you are tracking users you might as well have a dynamic SSH key distribution and user configuration for each server.

The two things that you do NOT HAVE TO WRITE is the MTA or the IMAP server.

Get to it Internet, (feel free to fork Notice). Integration WITHOUT conglomeration. And when you aren't making bricks, add in fax and SMS support.

 Or you could cop out and just use ATmail, (or mailpile if that becomes what I expect that it will.)

Wednesday, 31 July 2013

Outside of the box and in childgarden

As Sir Ken Robinson, (and The Bible) tells us, children can be an unusual source of wisdom. The one that impressed me the most was a researcher asking children how to save energy in the home. One reply was, "ban toast". This was remarkable for me because children usual love toast, (I know I do) and the clarity of logic when asked why, "because it is already cooked" seemed impressive.

I re-thought toast and found that I only need to toast things for two reasons.

  1. To deal with slightly stale bread, (or as the French would call it, "baget".)
  2. To make it easier to spread refrigerated butter
The first can be solved by re-purposing the bread through soaking it in pickling vinegar and using it to thicken soups. The second can be solved during winter by leaving the butter out and during summer by turning the fridge down, (and a variety of other methods, that sadly require thinking-ahead.)

This naturally lead to me wondering how to improve the refrigerator, and from that direction it seemed obvious that 10x50x1.2mm [0] strips of clear transparent PVC with individual right-angle clips could be attached in rows to each shelf to reduce the cold air flowing out of the fridge each time that the door is opened. The problem is that the things in the door might have reduced cooling and if the cooling required circulation then hanging walls of PVC at the front of each shelf might hamper performance. 

[0] Not all shelves are the same height, so 10x50, 10x100, 10x200, 10x500 mm should also be available.

It is inflammable iregardless of what you think






Wednesday, 24 July 2013

Pi.ranoid Raspberry Pi + Zoneminder

I'm surprised that I have not seen anyone with an image of zoneminder for raspberry Pi. What is even more surprising is that piranoid hasn't been picked up as the obvious name.

I'm now working out if I need an entire slice of Pi or something simpler like an Arduino to automate the watering in a friends greenhouse. The Tinkerforge bricks also look interesting.

My plan requires measuring air and soil humidity, and ambient temperature and the amount of red and blue light. The long-term plan is to also compensate for the lower light levels in winter by enabling banks of red and blue LED's powered by a pico-hydroelectric system.

Add useful links in the comments.

Thursday, 18 July 2013

The Rice Matrix

Cooking rice. Millions of people, (mostly women) do this each day. Despite the 'lacking in vitamin A' issue, rice seems to be a good thing to eat.

A few years ago I started to notice that many people had: The way to cook rice. The thing that interested me was that none of them were the same method. After collecting a few more variations I've come up with a matrix. The two types of rice that are produced I refer to as sticky and long-grain. (If you cook long-grain long enough and without rinsing, it is possible to make it sticky.)

Rinse before ?NopeYup - cold water
Water temperature?HotCold
Rinse after?NopeYup - cold water

As you can see, there are a few different method and actions. Long-grain, (the method rather than the type of rice) could be rinsed with hot or warm water, but most of the people that I interviewed used water directly from the cold-tap.

Water temperature is the temperature of the cooking water when the rice is added, (hot usually means boiling).

Do you cover your rice or leave it open? Can you use less energy if you cook it on a lower setting with this a lid on the pan?

Do you cook in enough water for the cooked rice to swim or do you boil it dry?

This just covers those with a hob, (wood, gas, electric, halogen, induction, Aga) and a pan.

Then there are the risotto type rices and the pudding rices.
This does not include those who swear by their rice steamers, (I'm trying out a new one from Lakeland plastics that goes in the microwave - more on that when I have it.)

How do you do yours, and more importantly, why?

Youtube pumping

With the new HTML5 on a 2Mbps ADSL connection I've found, (in Chromium on an old laptop [0]) that I have to pause the video, (using the space bar [1]) to manually buffer the stream from time to time. I don't know why the browser can't do this automatically, but it feels like it is sleeping/waiting for the browser load to go down before it hits the network. (And when it drains the buffer it stops playing which triggers it to get more.)

I really look forward to HTML5 video, but it seems that there is still a kink in the browser to work out.

[0] 32bit Intel CPU@1.20GHz running Ubuntu Linux with 2Gig of RAM.
[1] it should be possible to disable "space-bar = page down" for pages that have HTML5 video, because paging down when you mean to pause is frustrating.

Wednesday, 10 July 2013

Writing environment

I do most of my writing, (both code and stories) in Linux. I launch screen with my custom screenrc and then use vim. In my ~/.vimrc I have:

function! Doc()
  set linebreak
  syntax off
  set spell spelllang=en_gb

function! OffDoc()
  set nolinebreak
  syntax on
  set nospell

" so ,doc (comma-d,o,c) typed into command mode runs the Doc() function
map ,doc :call Doc()<CR>
map ,offdoc :call OffDoc()<CR>
map ,od :call OffDoc()<CR>

so when I'm not coding I type ,doc and vim turns into the perfect writing environment for me, . o O ( because I cost it!)  [0]

When I find myself in some strange non-linux OS I use notepad++ (which even works rather well in WINE.)

I should probably use ( just confuses me. Yes I'm that old.) If you are thinking of writing an editor for your website I would strongly suggest that you look at CKEditor first. (Yes RT4.0 I'm looking at you!)

Remember that you can implement an entire wiki, (more than a text editor) with one line of perl:

[0] Yes this is my L'Oréal joke.

Tuesday, 9 July 2013

Getting It There (git)

git is very interesting. Shockingly easy to learn and clearly going to take over RCS. I think that it is worth noting that like any tool it takes training and discipline. Being decentralised and with branches costing nothing, (now that I've found meld to help with the merges) you can run into problems while learning it.

To that end I think that we should have an acronym competition that highlight the pitfalls of having so much power.

Generating Innumerable Tangents (branch clone)

Generating Invulnerable Tangents (can't fudge the SHA1 tree chain)

Do you have any more?

Wednesday, 3 July 2013

Software Available for Humanity

I've mentioned previously in this blog many impressive pieces of free software. Now while the Google and RMS argue over the meaning of free I proposed a new project: Software Available for Humanity or SAfH, (pronounced safe).

This would be a cross between (yes I'm showing my age) and PortableApps with a dash of
Each program would be one of the best at what it does without any agenda. I don't mind which opensource licence it is under but using BSD/MIT/LGPL would count as being better than GPL, (though still good) and hence rank that piece of software as being better.

Criteria: Nothing bad and certainly no evil.
Listing: platform, Program, function, md5sum (why aren't hashes in a standard form of filename.hash with the string inside of  "{md5}blah filename" so that we can build binary verification into download applications?), version, filename, date added, date replaced, [ list of download locations], author(s), homepage, labels.

Then when someone is looking for "a win32 program to remap keys" they will end up with

{"platform":"win32","prog":"SharpKeys","function":"Remap keyboard;Make certain keys on a keyboard act like other keys", "hash":"{sha1}6b19f64804fb8df3a246303e46307f013993f544", "file":"sharpkeys35.msi", "version":"3.5", "added":"2012-02-14 09:00:00", "url": {
}, "author":"RandyRants", "homepage":"", "labels":"keymapper, remap keys, win32, act like another key"}

If we have an API (or just an invisible json string in the HTML) then it should be easy to have a sort of application that searches for and installs software. Mix that with SpiderOak, (to remember which applications to install) to backup your data, then you could rebuild your desktop with a single click.

I've noticed that some good honest decent programs in portable apps, like IObit Unlocker Portable don't mind where you install them, (and finally have a sane default install path of CWD) while the corporate lures object to being installed in C:\Program Files\. This would fall into the category of "bad". Not exactly evil, but not "for Humanity". 

When it comes to functions, there is almost always a free version that is faster and certainly with 99% less, "you could 'upgrade'" be that "we have a new version", (Google Chrome, I'm looking at you doing your "silent updates" that seem to freeze older computers) or "buy the full version to get that additional functionality".

The problem now is linking the users, (humanity) with the best free options.
(And listing the functions that are yet to be written.)

Do you have a killer app that you with was on the list? I would not mind how specific its functionality because the search would only match to the users requirements, (so a single scripts that just removes win32.trojan.ransom.834 would be fine if it did just that and nothing else.)

I would start with (for win32):

Wednesday, 19 June 2013

Speeding up Exim

Hit a strange "feature" in exim today:

# time exim -bt $(whoami)
  router = virtual_local_mailbox, transport = virtual_user

real    0m0.047s
user    0m0.016s
sys     0m0.010s

47  milliseconds. Not too shabby.

# time exim -bt
  router = dnslookup, transport = remote_smtp
  host []  MX=5
  host [] MX=5
  host []  MX=5
  host []   MX=5
  host []  MX=5
  host []  MX=5
  host []  MX=5
  host []  MX=5
  host []  MX=5
  host []  MX=5
  host []   MX=5
  host [] MX=5

real    0m45.128s
user    0m0.010s
sys     0m0.016s

45 SECONDS? to do a blind-test? I hear DNS alarm bells in my head. So it is off
to: we go.

exim -v -d -bt 

(you could inject a test message if you want to do it for real:
exim -d  
This is a test
exim whipps through to
--------> dnslookup router <--------
and then grinds to a sticky slow as it does
2a00:1450:400c:c03::1a in " :"? no (end of list)
DNS lookup of (A) succeeded in " :"? no (end of list)
DNS lookup of (AAAA) succeeded
2a00:1450:4001:c02::1b in " :"? no (end of list)
DNS lookup of (AAAA) succeeded
2a00:1450:4008:c01::1a in " :"? no (end of list)
DNS lookup of (AAAA) succeeded
2a00:1450:4010:c04::1a in " :"? no (end of list)
DNS lookup of (AAAA) succeeded
2607:f8b0:400e:c03::1b in " :"? no (end of list)

Now just as bcrypt deliberatly slows things down, this could be considered an anti-span feature, but I want my mail servers to deliver mail rather than to slow things down.

So lets look at that part of my exim config:

  driver = dnslookup
  domains = ! +local_domains
  transport = remote_smtp
  ignore_target_hosts = :

so is it catching on ignore_target_hosts, (that is important), or just the DNS lookup.
[ Why not test ignore_target_host on the first MX host, try to deliver, and if we fall back to the next host, _then_ do the next ignore_targe_host test? Because exim calls its ROUTERS (in order) and _then_ calls the triggered TRANSPORT. This means that exim wants to verify _all_ routes in the ROUTER /before/ any transport. ]

Lets REM that out... and it _still_ takes 55 seconds! So just

DNS lookup of (MX) succeeded
DNS lookup of (AAAA) succeeded
DNS lookup of (A) succeeded
DNS lookup of (AAAA) succeeded
DNS lookup of (A) succeeded
DNS lookup of (AAAA) succeeded
DNS lookup of (A) succeeded
DNS lookup of (AAAA) succeeded
DNS lookup of (A) succeeded
DNS lookup of (AAAA) succeeded
DNS lookup of (A) succeeded

is the bottle-neck. So how do we sniff out the name server/network?

As I've mentioned over here:

tcpdump -lvi any "udp port 53" 2>/dev/null

So if exim -bt is slow for !+local_domains : !+relay_domains, (i.e. remote/DNS domains) but its usual fast speed for
exim -bt $(whoami)
then you have a DNS problem, (or feature, depending upon your exim queue).

I also installed unbound on port 43 and added nameserver to /etc/resolve.conf (as the only nameserver) and that took exim back up to full speed. I'm not naming names, but XXX.XXX.80.26 should not be in handed out via DHCP for NetworkManager if it isn't going to do resolving.

SpiderOak 5.0.1 on CentOS 6.4 x86_64

SpiderOak is the best fit for backing up my life. First off they do security properly, (unlike the laughable "most popular" that I would "drop" in a heartbeat.) Secondly they seem like nice people, ( could go on...)

I wanted to add some of the files on my personal server that happened to be running CentOS 6.4 (the 64bit version). So I downloaded the RPM from the SpiderOak site and was hit with some cryptic python error[1]. ( I only have ssh access to that server, so I was only trying from the command line.) I expected that my version of python was missing something, so I tried to fix that. In the end I removed SpiderOak x86_64 and installed the 32bit version... and it worked![0]

Not sure why the 64bit version failed, but as long as it is secure I'm happy, (and when it "failed" it did so securely which is the most important thing.)

[0] well there were some errors:
Synchronizing with server (this could take a while)...
Error setting attribute: Setting attribute metadata::custom-icon not supported

but after Ctrl+c
SpiderOak --include-dir=/var/lib/mysql
seemed to work
SpiderOak -v --batchmode
did the job (the -v is useful to give you confidence as to what SpiderOak is doing. i.e. it is for you; SpiderOak will be fine without it.) 

[1]alexx@www ~$ SpiderOak --setup=-
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "", line 128, in <module>
  File "", line 12, in <module>
  File "", line 60, in <module>

Tuesday, 18 June 2013

selective rsync cracked --precurse-parents

We all know that rsync is one of the elite unix programs. It has no equal and it is so well written and so powerful why would anyone try?

So what is my problem?

I want to back up /var/lib/mysql/ and /etc/pki/ and I want to do it recursively so that I recreate the actual path, (none of that incestuous relative stuff of me!)

What I /think/ I'm after is:

rsync --precurse-parents -maPAX \
--filter='+ /var/lib/mysql/**' \

--filter='+ /var/www/sites/*.org/**' \--filter='+ /var/www/sites/notice.*/**' \--filter='- /**' \
--filter='- *' \--rsync-path='sudo rsync' 'rsync@server:/' /var/backup/server

Where the --precurse-parents   would be like  --prune-empty-dirs
 but would include the parent dir /var and /var/lib because of /var/lib/mysql while excluding /var/* and /var/lib/*.

It is something that I've fought with for over a decade. I've written perl scripts to solve the problem. I've written bash scripts. I've even been crazy enough to read the documentation, (man rsync), but it wasn't until today that I understood.

about 83% of the way through the man is:

       include/exclude patterns get applied recursively to each subcomponent’s full name (e.g. to  include  "/foo/bar/baz"  the  subcomponents  "/foo"  and
       excluded section of the hierarchy.  This is particularly important when using a trailing ’*’ rule.  For instance, this won’t work:

              + /file-is-included
              - *

       This  fails  because  the  parent  directory "some" is excluded by the ’*’ rule, so rsync never visits any of the files in the "some" or "some/path"

              + /some/
              + /some/path/
              + /file-also-included
              - *

And that solved the problem for me:

rsync -maPAX \--filter='- *.swp' \--filter='- .git/' \--filter='+ /var/' \
--filter='+ /var/lib/' \
--filter='+ /var/lib/mysql**' \

--filter='+ /var/www/sites/' \
--filter='+ /var/www/sites/*.org/' \
--filter='+ /var/www/sites/notice.*/**' \--filter='- /var/www/sites/*' \
--filter='- /var/www/*' \
--filter='- /var/*/*' \
--filter='- /var/*' \
--filter='- /**' \
--filter='- /*' \--prune-empty-dirs \

I think of this as, "include /var/ {so that rsync can see /var/www}"
"include /var/www/sites/*.org/ {include all of the .org sites}"
"include /var/www/sites/*.org/** {and the files+dirs of those .org sites}"

The mysql line includes the desired dir and everything in it, but would also match /var/lib/mysql_archive_do_NOT_backup, so it is a little more risky.
So each time rsync has to chose it goes through the whole filter form the top down and includes/excludes things that it finds, and if it hasn't included /var/www then /var/www/sites is _never_ going to match. The usual advice is to try the following:
 rsync -maPAX \
--filter='+ */' \
--filter='- /var/www/sites/*.org/**' \
--filter='- /var/*' \
--filter='- /**' \
rsync@remote ~rsync/backup/
but I think that the first filter line
has the hardest implication to comprehend.

rsync -mnavvPAX  from to
is really helpful (the -nvv does a dry-run and gives additional info.)

 This would then be:
rsync -dwim --filter='+ /var/www/sites/*.org/**'  server /var/backup/server/
I'm sure there is still a better way to get rsync to precurse-parents, as it were, but I'm happy with this solution, (until some kind person adds a comment suggesting an even easier or quicker way to do this.
 [dwim = Do What I mean; not a real rsync flag]

Monday, 1 April 2013

Easier before you are dead

Dead men tell no tails, 
(of what their passphrase is, once they are gone.)

I think that most of us can agree that it is hard to get much done after you are dead. I overheard two people talking about death and how hard it can be to clean up after people. The example that one gave was, "how do you take down their social-network?". This is a good question.

I introduced myself and asked if they wanted to know the answer. They seemed to, and that is what I'm going to share with you now. Before I do that there are just two things that I need to check.

You have installed a proper backup solution and that you have something like keepass in which to store all of your private details. (I'll wait while you download and install them - I'm not going anywhere... unless I've died.)

So now let us imagine that you have a nice long, but memorable phrase that you use to lock keepass. You can use something called Shamir's Secret Sharing Scheme to do a little mathamagic.

SSSS lets you take a number, (lets pick 15) and create a number of shares, (lets say five). Each share is useless on its own, but, (and this is the clever bit) you can chose how many have to be combined to recover the original number! (If your mind hasn't been blown then read on.)


The number of shares required is called the quorum, (in this case 3), and it does not matter which of the shares are used as long as they add up to the quorum, (you can only use one share once.)

You can test this at paste in any three of the above and press combine. (If that site goes offline because not enough people have contributed or because someone has died, then there are ssss programs like ssss-split that you can download and use.)

So how does this help? Well the words that you are reading right now are on a computer, (unless you printed this out) and words, for a computer are just numbers. So it is possible to put your passphrase through SSSS and create a share for each of the people that you trust. This means that, after you die, they can get together, (or just email it to each other) and recreate your passphrase. I gave one share to each of my five best friends, and four shares to my lawyer. The quorum that I set was 7, so my lawyer will need any three of my friends to recreate my keepass passphrase, (I hope that no more than two of them die in that car crash with me!).

I change my keepass passphrase on the 1st of every month, so how do I get update to my lawyer?
I don't!
I have passwords.txt inside of a truecrypt encrypted container and on the first of every month I update it with the latest dump from keepass... (and then I check it in because I have the inside of under GIT version control.) - so my friends have to find, which is in my Documents and has a link from my desktop on a special computer that is encrypted with a passphrase that is recorded in each of their keepass, (and if they forget I have it mentioned/explained in my will.) 

 The only problem with this is that you will not be able to do this once you are dead, and you do not know when you are going to die, (so DO IT NOW!.)

The reason that SpiderOak is so good, (other than the encryption done properly) is that once it is installed you can forget about it. That means that it makes life easier rather than harder. Moving over to keeping important data in keepass might seem hard for those of you still silly enough to use one password for everything, but those that have a piece of paper with notes on should at least think of keepass as a good backup for when that paper is lost, (though you might be tempted to keep that piece of paper after you have entered all of your existing passphrases and bank details into keepass, but DON'T! It will only lead to problems in the long-run and possibly problems for those that you leave behind.)

Things that would be helpful.

1. An easy and encrypted way to transfer keepass entries to someone else. I'm thinking some sort of PGP based module in keepass that would encrypt and send one, (or a group) of entries to a remote address, and an easy way to import them. I don't mind holding onto a SSSS share for a friend, but I'd like to be able to import it as easily as cut-n-paste.

2. A nice way for a lawyer to store all of their clients shares. I've printed out the shares for my lawyer and they are part of my will, but so far my lawyer has resisted installing SpiderOak, (he is still running on windows 98!) because he does not trust it, (or it might be that I failed to explain how impressive it is to him - I'll have another try.)

Friday, 1 March 2013

2013 first quarter amelioration







Thursday, 21 February 2013

Doing an update and it spits out:
Error reading from server. Remote end closed connection [IP: 80]
Fetched 2,422 kB in 23s (104 kB/s)
W: GPG error: raring Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 32B18A1260D8DA0B
W: Failed to fetch  Error reading from server. Remote end closed connection [IP: 80]


(happened to be ubuntu 12.Unity_sux).

Google found the solution for me on  but as it was not the first link I wanted to help bump it up, (if linking to a page still does that) and thank (I'm guessing from whois) Simon Pearce, for having the perfect reminder well laid out; thank you.

gpg --keyserver --recv 32B18A1260D8DA0B && \
gpg -a --export 32B18A1260D8DA0B | sudo apt-key add -

Not a gpg function that I perform on a weekly basis, so having a note can be helpful.

Sunday, 10 February 2013

People should stick to what they are good at.

Like ending a sentence with a preposition ;-) If people should stick to what they are good at, the next question should be, how do we find out what we are good at? (Becoming good at something is simply a matter of basic training and then daily practice + a little aptitude - it is hard to become good at something that you do not like.) A few years, (about seven) you will have mastered it.

A few days ago I found an air traffic control simulator. Not only is this very cool, it made me wonder why we don't have a game for every job.

I am reminded of the games ConflictInheritance - Simon Hessell Software and the kingdom simulator on the BBC Microcomputer Welcome! disc. These were relatively simple, but I found that I could play them for hours.

Children like games, so why not have a "being prime minister" game and a, "being a fireman" game. Each game would have to test aptitude while also trying to convey some sense of the real experience, ( I leave this up to the game developers.)

Another example of learning through fun would be Robocode. The idea is to build a robot tank, and have it fight with other tanks. The skill learnt, at the same time, is programming in Java. I believe that all programming is related in some way, so even if you wanted to become a objective-c programmer or a python hacker, having played this game when you were younger would help.

We should start wars where nations have virtual wars with autonomous robots that actually kill each other, (what could possible go wrong with that?) Though there should be a significant bonus for achiveing a win with the minimum cost, (so that it isn't just the most well funded that wins - or we will end up with the current mess that global leadership is in.)

[update] If you look into the Internet long enough, it will look back into you. Well not quite, but it seems that the potential for this already exists, (and I am a fan of using browsers as the frame.)

Disclosure: I've never been world-class at anything, but I've known many people that were.

Saturday, 26 January 2013

50 shades of Spartacus

I see that the Spartacus train has got to the third season. I wonder if this will damage the sales of 50 shades books.

Monday, 7 January 2013

New CMS - Notice::Pages

Have you ever wanted a CMS to manage a website [0] that creates static pages, (either with or without a template) ?

Do you want that template to be just HTML with no micro-language, (other than HTML) ?
Do you want that template to have the option of having as much CSS and javascript as you, (or they) want?

Do you want someone, (or even a team) to be able to write and edit the pages and only the Editor, (or Editors) to have the power to actually publish, (set live) the pages that are ready, (and even un-publish them.)

Welcome to Notice::Pages A small perl module build on Notice

Notice::Pages creates flat-files and integrates with an instillation of Notice to create the static portion of the website. (All of the dynamic parts and those that require authentication reside the far side of Notice::Login.)

Deceptively simple, but infinitely flexible, (and with a touch more development it will have the option of a separate template for each page and one static site per account - so that Notice can optionally host multiple sites at the same time.)

Using the deceptively powerful CKeditor, Notice::Pages is light-weight, ready to use, (and looking for a strap-line - and some feed-back.)

[0] or more likely you want the customer, (your friend/parent/sibling) to manage their own website.

You can't move


The new Zeno's paradox

Imagine an infinitely long conveyor belt that is one kilometre wide. On each side of the conveyor belt is flat uninterrupted terrain of the same material as  the belt. (The belt is also empty east of where our story is set.) On this conveyor belt we find two twins, (they are each others twin, not one from two pairs of twins.)

They grow up on the conveyor belt until they are twenty-one years old. They are, (by the miracle that improbable does not equal impossible) remain identical to this day. To celebrate their birthday they decide to have a straight line drag-race. They have two almost identical cars, except that one has a maximum speed of 2 kilometres per hour and the other has a maximum of 4 kilometres per hour.
(Not the most exciting of drag-races, but it makes it easier to remember the numbers.)

The starting blocks for these two vehicles is a contraption at the edge of the conveyor belt and suspends the slower car one millimetre above the belt, (no the wheels are not touching the belt), and the other one millimetre above the surrounding flat land. Neither car can turn left or right and the starting contraption, (that will drop both cars at the same time), sends them off parallel to the edge of the belt. (The wheels have been constructed so that they have perfect friction and both will start at their maximum speed.)

The belt is moving east, (always has, always will) at exactly 2 kilometres per hour, (this is the direction of the race).

You are friends with one of the twins and she asks you which car she should pick.

Those of you that know v1+v2 != (v1+v2) will tell her to pick the 4 kilometre per hour car as the 2 kilometre per hour car will be going ever so slightly slower than 4 kilometres per hour car.

So if this happened in a galaxy that is already moving at the speed of light, would they be able to move at all?

Wednesday, 2 January 2013

Combined Taichi

Combine all Taiji postures into one form

I think of myself as a fan of the Yang family style, but in reality that is the branch with which I am most familiar. I also like the Beijing (24) form. So my question to those that are familiar with two or more styles, is:

Where and how would you combine the elements from one style into the other?

I imagine starting with the Beijing form and then adding the missing elements from the Chen, and then the Wu, (and lets face it - Bagua while we are at it; yes I am saying that Wu contains some Bagua). Then anything that is left from the Sun/Shoon, (even more Bagua plus Xingyi), and Wu/Hao styles, (and even the missing elements from the Yang style long-form). This would produce a medium length empty hand form of between fifty and seventy movements.

Start at the beginning!

The reason that I would not start with the Chen form and add the others, (despite Chen being the original) is that a lot of work went into condensing the Yang long-form from about 85, (108 if you like) movements down to 24. 

What would be the benefit of this?

Despite this thought experiment basically recreating the 42 (competition) form, or the history of Sun/Shoon style, I feel that analysis of the benefits of taichi [0] would be helped if we created a form that not only flowed from one posture to the other but over the course of the form it also flowed in and out of upright and inclined and from large frame to compact, (though not necessarily in that order, and probably changing every three or four postures.)

Xingyi < Bagua < Taichi

If that idea isn't contentious enough, I would also postulate that, as with Sun/Shoon that evolved from a person that had already learnt Xingyi and Bagua, (and despite that being the direction of evolution), (from Xingyi to Bagua to Taichi - how many practitioners of Bagua will be inflamed by the idea that Taichi is more evolved that their system? ;-) ) an individual may be the result of evolution but it is not hampered by the past. Put another way, those elements that are superfluous can be replaced by those that are superior; and by starting from Taichi and only adding the most distilled parts from the past we will not end up rejecting the past simply because it is older.

I'm comparing apples and oranges?

Most people will benefit from learning the Beijing form; this and Yang style are the most popular over the world. The other branches focus more on martial application. If you have steam coming out of your ears or find yourself shouting at the screen, "c does not imply causation" and Yang style is a fighting style then you are in agreement with all the masters. The only argument that feels valid, in regards to the differences between the branches is that a particular frame may match your athletic abilities and body morphology. 

"Taichi does not require a warm-up"

I often use some Qigong as a warm-up before I do one of the forms. When I learnt a short-form from a nice chap called Ian, (in London) we often did a particular qigong before working on the form. I can picture some of that qigong, but I don't remember it. With that frustration I asked on a Tao forum if anyone had been in those lessons or had any idea what it might be. One of the first responses, (how ever well meaning it might have been) was along the lines, "Taichi does not require a warm-up". I was amused that they had totally missed the point of my question, despite the validity, (or otherwise) of their response. If we added Qigong to the start of the form, (8 piece of brocade?) then we could be sure that it did not require a warm up. 

Has this idea got any legs? (Would it work?) The answer is probably not, but analysing how to combine styles is probably a good experiment for the more advanced practitioners. You can add your list of moves or stories of success in the comments.

[0] After the death of each Taichi master, I wonder what they died from, how long they lived and the quality of their last ten years. There should really be a table somewhere.

About this blog

Sort of a test blog... until it isn't