Category Archives: Programming

On the Importance of ‘Technical Debt’
















[3] Such as ‘Organizational Debt‘.





The Mysterious Case of the Regex Dot

So, I’m in the middle of organizing my photos into folders, something more useable than the default Photos application on Mac[1].

While trying to count the number of photos/videos[2] in each subdirectory in my …/2018/ folder:

$ time find * |grep IMG|grep -o ‘^[0-9][0-9]/.’|uniq -c
22 04/0
3297 05/1
104 05/2
100 06/0
1830 06/2
2040 10/2

I first tried the supposedly logical:

$ time find * |grep IMG|grep -o ^..|uniq -c|head
1 04
1 /0
1 1/
1 20
1 18
1 04
1 01
1 -0
1 00
1 41

Interestingly, grep (and/or the OS) seemed to be taking the front off of each line, and then putting it back into the STDIN hopper for the next call to grep.

As this was not doing what I expected (nor wanted), I tried:

$ time find * |grep IMG|grep -o ‘^[0-9][0-9]/’|uniq -c|head
1 04/
1 01/
1 04/
1 01/
1 04/
1 01/
1 04/
1 01/
1 04/
1 01/

Which, while better…

$ time find * |grep IMG|grep -o ‘^[0-9][0-9]/’|uniq -c|sort|uniq -c
22 1 01/
22 1 04/
3501 1 05/
1930 1 06/
2040 1 10/
3297 1 13/
104 1 23/
1830 1 24/
2040 1 27/

…gave me too many results by about a factor of two, and somehow found 27 months in the year.

I quickly figured out that while parsing mm/dd/yyyymmdd-hash/IMG_[0-9][0-9][0-9][0-9].[FILETYPE], this particular grep/OS combination will happily grab the ‘mm/’, and then also grab the ‘dd/’. This habit, while charming, does not solve my problem.

After google searching proved unfruitful, I decided to try:

22 04/0
3297 05/1
104 05/2
100 06/0
1830 06/2
2040 10/2

and it worked!

I was stumped, until I figured out that the issues that I had been seeing before were entirely because grep was finding results at the start of the newly chomped string, and that by chomping part of the next ‘match’, I was stopping grep from finding any more matches.


[1] Right now, when Photos organizes photos, it puts each photo into its own folder, based on year/month/day/yyyymmdd-hash, which makes it super-annoying to use anything about the Photos app, which is super-slow and annoying to use.

[2] The images are all in the format ‘IMG_[0-9][0-9][0-9][0-9].[FILETYPE]’, where FILETYPE can be ‘PNG’ (screenshots), ‘JPG’ (camera pictures), ‘MOV’ (camera movies), ‘GIF’ (saved .gifs), or perhaps some other recognized image format.




import wave
import struct
import math

WAV_FILE_LEN = SAMPLING_RATE * 1 # 44.1KHz sampling rate, 5 seconds
MAX_AMP = 32767

output_file ='test.wav', 'w')

for i in range (0,WAV_FILE_LEN):

print data
packed_data = struct.pack('h', data)




*Also 100Hz.

Interview Questions: Types of Coding and Algorithm Questions

Part of a continuing series on Interviews and Interview Questions.













[1]Hat tip:

Interview Questions: Other
















Interview Questions: Technical









So what is an interviewer looking for in these questions?




Generally, the interviewee is given a problem statement:








Write down assumptions
Draw a big diagram
State the overall algorithm
Write down the solution, while talking about it


“Of All the Things I Miss, I Miss my Cache the Most.”

She was in the zone. It had taken two hours, half a RedBull (she would be paying for that later), and pissing off that guy who always seemed to want to talk longer than a conversation.

Free, flying through the code. There really was nothing like it.

She was working on a new DB caching layer for their server-side app. It was one of those ‘augmented reality’ games, but for corporate training. It still felt good to work on it though, to coerce the bits to bend to her will.

“Achievement Unlocked: You have met five new people in one day!”

Ugh. It sounded terrible, having to meet so many people all the time. To have to spend all that time and effort to convince them that the correct answer to a problem was, well, correct. If only they would just *see*.

But she had given up hope of being able to open peoples’ eyes. Give her code, or a nice juicy math problem, and she’d be content for hours, sometimes days.

*rumble* *rumble*

“Time to eat”, she thought to herself. She gets up, to go to the kitchen. Nyancore streams from her discarded headphones. As she turns, you can see the t-shirt she’s wearing says:

“Of all the things I miss, I miss my cache the most.”

She looks towards you and says “You can’t fault me for that.”

She laughs to herself and continues on her way.

“Senseless Juxtaposition of Wildcards.”

He had to admire the the gall of the programmer who wrote the error messages.

“Senseless Juxtaposition of Wildcards.”

It might as well have said:

“Grow a brain!”


“Try listening to classical music.”

But then it got him thinking…

What would be a senseful juxtaposition of wildcards?

First, we would have to make a list of possible wildcards:

The ‘standard’ wildcard character, specifically referring to a character is the question mark, ‘?’. Generally standing in for any one of some set of things (or in Perl, 0 or 1 of a thing).

The ‘larger’ wildcard character, ‘*’, which stands for any number of something (including 0), sometimes expressed as ‘%’, if you’re speaking SQL.

The ‘even larger’ wildcard character, ‘…’, which is like a recursive ‘*’.

But could there be something larger still? Something which climbs the directory hierarchy in the oppsosite direction, perhaps? Something which can make it past all of the automatic filters, but is clearly wrong? Something like typing ‘NaN‘[1] into a number field box? Something which steps outside the usual boundaries, like Thiotimoline?

In a biological context, there are entire alphabets of more-and-less-specific wildcards.

So, knowing all of this, what would be a senseful juxtaposition of wildcards? Something like ‘**’, or ‘?*’, or ‘*?’ would be meaninglessly equivalent to ‘*’.

You could attempt to mix SQL with bash-isms: “WHERE ID LIKE ‘%*’ “, showing that you expect an SQL character string followed by a bash character string, but that is again non-sensical.

Maybe it would have to be something like ‘hello??????'[2], to say that there are 6 characters of some type after your ‘hello’.

But there it was. The senseful juxtaposition of wildcards… bash statements inside command-line SQL statements.

That was it! But he had to think. How would he use this?

[1]And like the link says, you really don’t want to confuse it with NaN3. You really don’t want to confuse *anything* with NaN3.

[2]Or ‘hello……’.

Adventures in Mobile Phone Resurrection

My day, in a nutshell:


0) See that your phone may be having issues. No time to spend the week fixing it. Tape[1] it up and take it to Burning Man. Wait 10 months for the issues to become serious. Go on vacation. Drop it in the airport on the way there. Nursemaid it through the vacation, trying to read through the horizontal lines of ‘VGA cable is partially detached’. Drop it on the plane on the way home.

0.5) Take it in to the store. They say they’ll replace the phone for $100, but the data won’t be transferred over. I buy a new phone and go home to check my backup situation. (All my photos and videos are fine, it’s my notes and TTD lists that I’m most concerned about.)

0.7) Get home and find out my last full backup sufficient for a ‘Restore’ is about 10 months old. For some reason the ‘Sync’ doesn’t actually sync any useful amount of data, and there are no useful ways to gain finer control of this (I’m assuming) without jailbreaking the device.

0.8) Restore the device using the old backup. It’s really odd to see your last messages with someone that are 10 months old. Resolve to transfer over the rest of the data somehow…

[Next Day]

1) Start backing up the old phone. Discover that your phone needs some number of tens of gigs to do a full backup, and your computer only has 11GB free.

2) Rummage through your hard drive, using suggestions from helpful sites, finding another 15GB.

3) Figure out that 25GB is again not enough. Start taking a closer look at that 40GB backup from 10 months ago. Look at the directory, noticing that it contains about 40k files, each named with a 40 character hex hash. Try uploading it to back it up. After about half an hour, calculate that it will take 15-20 hours. Try tar -czvf to reduce the number of files. this doesn’t significantly help the upload time.

4) Decide to bite the bullet and delete the old backup. As there are too many files in the directory for ‘rm *’ to work, start with ‘rm 1*’, through to ‘rm f*'[2].

5) Start the backup of the old phone again, it finishes, and I start the restore to the new phone.

6) After waiting for a while, the phone has reset, and it gives me the option to ‘restore from backup’. More waiting. ‘The backup was corrupt or not compatible with this device or device version.’

7) Try again. More waiting. Once again having the new phone factory reset to start the restore. Once again: ‘The backup was corrupt or not compatible with this device or device version.’

8) Look in the directory to see if there’s something obviously corrupt. Wait a second…Some of those files starting with ‘0’ are from 2015…

9) Move all of the old remaining files to a new folder. Try the restore again. ‘The backup was corrupt or not compatible with this device or device version.'[3] Realizing that I had missed other files (not starting with 0-f), or that some of the 0-starting files had made their way into the list of files for the new backup, I delete all of the files in the directory this time.

10) Full backup. Full restore. Full day.

[1]Duct tape, of course. My little friend was suffering from an ‘expanded battery’, which eventually became bad enough that the screen became separated from whatever was feeding the screen data. Two drops later, it was unfixable, thankfully still okay enough on the inside to back up.

[2]You may notice something here.

[3]Interestingly, (from behaviour, not from looking in the files), the backup program (iTunes) blindly adds new files to the list of hashed files, and probably adds them to a list somewhere in that directory. It apparently doesn’t do much checking of the backup until it tries to restore it somewhere.

Running A Sprint Planning Meeting

It’s the little things that sometimes make a difference. When I was teaching standardized test math so many years ago, I noticed as I was drawing problems on the board, all the little habits that I had picked up. Habits which make solving problems easier, habits which reduce the chance for error.

Things like the curve on the leg of the lower-case ‘t’, so that it doesn’t look like a ‘+’. Curving your ‘x’ so it doesn’t look like a ‘*’ sign.

I think some of this (probably sometimes annoying) attention to detail had carried over to Sprint Planning meetings[1].

Planning Poker is a method for a group to converge on a time estimate for a task or group of tasks. There are a number of ways to do this. The ‘canonical’ way we were taught to do this was to use Fibonacci-numbered cards (1,2,3,,5,Eureka!). This involved a discussion of the task(s) to estimate until everyone had a reasonable idea of their complexity, then each person would choose a number estimate, all of which would be revealed simultaneously, to hopefully reduce bias. The discussion before estimation would not include estimates of how long things were estimated to take, to also try to reduce bias.

While we were running our planning meetings, I noticed that we would start to slip away from this ideal, perhaps because certain things were not important, perhaps because we didn’t see that certain things were important. For example:

We moved from cards to apps, and then to fingers. Using apps for estimation is less annoying than finding the cards each time, but fingers are even faster to find. I/we tried to get around the bias effect by having everyone display their fingers at once, and that worked reasonably well. Even making each person think about their estimate before display can help a lot with reducing the impact of what others might think of them.

One thing I tried which never really caught on when other people were running the meeting was saying ‘A,B,C’ instead of ‘1,2,3’, with the idea that it would be less biasing on the numbers people were choosing. (This may have mostly been an impression of mine, as the moving of the estimate from a mental number to a number of fingers may cement it in a slightly different mental state…)

If one is not careful, and perhaps somewhat impatient in meetings[2], one can start suggesting estimates before they are voted on. It can take considerable discipline and practice to not do this.

Another thing I noticed was how difficult JIRA was to use when one is not practiced in it, especially in a room with many people watching. Something that any experienced[3] demo-giver would know like the back of PowerPoint’s hand.

That’s all I have for now. For more minutiae, tune in tomorrow!

[1]For those of you who have not had the pleasure, these are the meetings at the start of an iteration, where the team sits down in a room, estimates a bunch of priority-ranked tasks, and decides (generally by consensus) how many of them they will commit to getting done in the next two weeks. Like all meetings, they can be good or bad, and the meeting chair (I feel) can make a large difference.

[2]I am probably as guilty of this as anyone. I would recommend Randy Pausch’s ‘Time Management‘ for those who feel similarly.

[3]Read: ‘Battle-scarred’