[12:59] <Umeaboy> Hi!
[12:59] <Umeaboy> This guide is out of date: https://wiki.ubuntu.com/ARM/RootfsFromScratch
[12:59] <Umeaboy> What can I use that's more up to date?
[13:00] <Umeaboy> I have installed live-build and qemu along with deboostrap.
[13:00] <Umeaboy> Because the sudo rootstock won't work as there is no rootstock package in Xenial.
[13:01] <Umeaboy> That's a project that's been dead for ceveral years.
[13:02] <Umeaboy> ogra_: You around?
[13:03] <Umeaboy> I'm interested in making a chroot rootfs tarball to use with Sailfish OS.
[13:03] <Umeaboy> Ubuntu 16.04 is my main system, but I can use others.
[13:04] <Umeaboy> My goal is to see if I get less problems with a up to date chroot.
[13:04] <Umeaboy> I know that Trusty is getting EOL in August so I guess Xenial will be the next LTS to get a tarball.


[08:52] <apw> smb this thinkpad brightness/volume thing is all over the map ... most processes seems to get a hand in the handling of a simple screen update ...
[08:53] <smb> That was to be feared
[08:53] <apw> i think it has reached a level of complexity that a wiki page is required to contain the information
[08:53] <smb> So the entry is the key event, who comes next? hal probably
[08:53] <apw> well the key is generated by hal as its a fake key
[08:53] <smb> Hm, this sounds definitely like an asset
[08:54] <apw> then it goes to either power-manager or settings-daemon depending which it is
[08:54] <apw> then that talks to hal to change things and also send out a notification
[08:56] <smb> I assume power-manager for brightness and settings-daemon for sound
[08:56] <apw> yeah
[08:56] <apw> now brightness does have a passive mode already though it doesn't seem to work quite right
[08:56] <smb> Those are those things that would benefit from a picture...
[08:56] <apw> (passive being a 'just display the currentlt level on keypress')
[08:57] <smb> Ok, that is one step. So _whenever_ that works, something similar would be required for sound
[08:57] <apw> volume does nto as yet
[08:57] <apw> yeah ... something like that
[08:57] <apw> and that is basically what the maintainer is talking about ... sort of
[08:57] <apw> i guess i need to just dump it all out
[08:58] <smb> Yeah, that would be helpful, then print it and confront the right people at UDS
[08:59] <apw> heh yeah
[11:25] <apw> bah have spent too much time thinking about this on my own
[11:25] <apw> https://wiki.ubuntu.com/ThinkpadBrightnessVolume
[11:38] <cking> apw: wassup?
[11:39] <apw> just getting sucked into overanalysis of a problem
[11:39] <apw> was slapping self to get it documented and move on
[11:39] <cking> good wiki page by the way
[11:41] <ogra> just send a mail to evolution on each event ... then you get a mail notification at least :P
[11:42]  * ogra agress its a very good research
[11:42] <ogra> *agrees
[11:45] <ogra> apw, are you sure we will even use the alsamixer in karmic ? i would suspect us to migrate to pavucontrol at some point anyway (so it would completely be in software and on the highest level)
[11:46] <apw> ogra, no idea really.  if so someone else can figure out how to make pulse work with a hardware mixer :)
[11:46] <ogra> you cant, but you could do the same you do for brightness more easily with it i think
[13:05] <Riddell> linux dudes: what have you done with /usr/include/asm/errno.h from linux-libc-dev ?
[13:05] <amitk> Riddell: karmic?
[13:05] <Riddell> yes
[13:06]  * amitk looks at rtg ^
[13:06] <Riddell> kde4libs not happy "/usr/include/linux/errno.h:4:23: error: asm/errno.h: No such file or directory"
[13:08] <rtg> Riddell: start a bug and assign me to it. I've a conf call coming up in a bit, so I won't get a chance to look at it until after that.
[13:19] <Riddell> bug 373214
[13:19] <ubot3> Malone bug 373214 in linux "errno.h missing" [Undecided,New] https://launchpad.net/bugs/373214
[13:22] <elmo> Riddell: which architecture(s)?
[13:23] <Riddell> elmo: i386 and amd64 build failures have arrived in my inbox so far
[13:30] <NCommander> rtg, /usr/include/asm/* is missing (we ran into this problem on powerpc; obviously this isn't a ports specific issue anymore :-/)
[13:31] <rtg> apw or amitk: do you have time to look at bug 373214? I'm booked for the next bit, and it appears to be causing wide spread panic.
[13:31] <ubot3> Malone bug 373214 in linux "errno.h missing" [Undecided,New] https://launchpad.net/bugs/373214
[13:33] <rtg> NCommander: if its the same bug, its affecting kde4libs as well.
[13:36] <amitk> rtg: looking it up now..
[13:36] <NCommander> rtg, well, the problem is /usr/include/asm is empty which breaks anything trying to use kernel headers
[13:37] <NCommander> rtg, it looked like something managed the package in the buildd (neither myself nor TheMuso have successfully reproduced this issue, nor could we in a jaunty PPA)
[13:37] <rtg> NCommander: which would certainly explain '/usr/include/asm/error.h is missing from linux-libc-dev in linux_2.6.30-3.4' as described in the bug report.
[13:37] <NCommander> rtg, I just checked all the architecture linux-libc-dev
[13:37] <NCommander> its empty on i386, amd64, lpia, armel, and powerpc
[13:38] <NCommander> ia64 and hppa are ok, SPARC is current FTBFS
[13:38] <NCommander> *currently
[13:39] <amitk> rtg: so you just uploaded meta yesterday and the karmic kernel started becoming available?
[13:40] <amitk> ...in the archive, that is
[13:40] <rtg> amitk: yep, -2 is the first ABI that is referenced by linux-meta, but I uploaded -3 yesterday
[13:55] <NCommander> rtg, the broken header packages make it impossible to build linux and linux-ports in a karmic chroot ...
[13:56] <amitk> NCommander: could you put all the info in bug 373214
[13:56] <ubot3> Malone bug 373214 in linux-ports "/usr/include/asm/* is not present in linux-libc-dev" [Critical,In progress] https://launchpad.net/bugs/373214
[13:56] <NCommander> amitk, already there
[13:56] <NCommander> amitk, I'm pulling the buildd chroot, and fudge it as necessary to get the kernel building again to see if I can see what went wrong
[14:03] <amitk> hmm.. after all the headers moves in .28 and .29, there is no include/asm-x86
[14:03] <amitk> or rather all asm-<arch> has been moved
[14:05] <amitk> rtg: all include/asm-<arch> was moved to arch/<arch>/include/asm
[14:06] <amitk> NCommander: ^
[14:07] <amitk> essentially all include/asm-<arch> was 'moved' to arch/<arch>/include/asm
[14:08] <amitk> but arch/<arch>/include/asm is quite a bit larger than what include/asm used to be, so I am still studying it
[14:08] <NCommander> amitk, but isn't that how it was for 28 as well? (I have a include/asm here)
[14:09] <NCommander> amitk, I need to run to the DMV, I'll be back in ~45-60 minutes to continue debugging this w/ you
[14:09] <amitk> NCommander: it was partly done for .28. If you notice there is no include/asm-x86 in 2.6.28. I am checking if something was being fudged in the scripts
[14:10] <NCommander> amitk, not to my knowledge, but I may have overlooked something
[14:11] <cjwatson> shouldn't 'make headers_install' just work despite the rearrangements though?
[14:11] <cjwatson> the build log says 'INSTALL include/asm (52 files)'
[14:17] <ikepanhc> smb: ask a question, if I need the bug reporter to help me test the patch, shall I add some extra version string when make kernel package?
[14:19] <cjwatson> hmm, that's interesting, 'fakeroot debian/rules install-arch-headers' on jaunty seems to install a reasonable include/asm/
[14:19] <cjwatson> amitk: ^-
[14:20] <amitk> cjwatson: on the karmic tree?
[14:21]  * amitk tries it
[14:23] <amitk> cjwatson: curious indeed
[14:23] <cjwatson> amitk: yeah
[14:24] <cjwatson> hmm, and even in a karmic chroot
[14:24] <cjwatson> maybe that test is inadequate somehow
[14:26]  * cjwatson bootstraps his own chroot rather than using ronne's
[14:29] <amitk> hmm.. why is the i386 build passing ARCH=x86_64?
[15:01] <amitk> cjwatson: I don't get it. The buildds are not behaving like my local machine. fakeroot debian/rules binary-arch-headers works perfectly here...
[15:02] <cjwatson> just to save time, it's astonishingly rare for problems to actually be due to a genuine difference on the buildds
[15:02] <cjwatson> it does happen but not typically in the *middle* of a build like this ...
[15:03] <cjwatson> might be worth trying to match up package versions with those in the -3.4 build log?
[15:06] <cafetiere> apw switches to another machine for testing
[15:21] <cjwatson> amitk: no dice for me in a chroot with linux-libc-dev 2.6.30-2.3 either
[15:24] <amitk> cjwatson: I'm doing a full build here
[15:24]  * cjwatson tries downgrading gcc and friends to the versions in the build log
[15:26] <NCommander> amitk, any headway? (I see you confirmed that local builds do work correctly)
[15:26] <cjwatson> has anyone tried a PPA build?
[15:26] <NCommander> cjwatson, yes, on ports, works fine in the jaunty series
[15:26] <NCommander> no karmic PPAs yet
[15:26] <cjwatson> oh, of course, it breaks due to the new linux-libc-dev anyway
[15:27] <NCommander> cjwatson, I have a buildd amd64 chroot now, so I'm seeing if I can find what's causing the package to explode
[15:28] <cjwatson> this is weirding me the hell out
[15:28]  * NCommander notes this doesn't seem possible.
[15:44] <rtg> awe: sent email re: Broadcom update. I think I've got the 2 patches you mentioned. It at least builds correctly.
[15:46] <awe> rtg: thanks... i'll look at it in a little while.  i'm fighting to get voip configured so i can have a conf call w/asac.
[15:46] <awe> rtg: no more hp for me for now.  ;)
[15:46] <awe> rtg: i'm working as part of desktop for karmic.
[15:47] <awe> rtg: although i'll still work with y'all wrt to the oem kernel
[15:47] <rtg> awe: do you want me to ripple the BCM update up through Intrepid/Jaunty ?
[15:47] <awe> rtg: nah, i'll do it.  it's good practice for moi
[15:47] <rtg> awe: k
[15:49] <awe> course i may ask you to endorse my ubuntu-contributors MOTO app page in return.  ;)
[15:50] <rtg> awe: np
[15:50] <awe> cool
[15:50] <awe> i've worked more with y'all then anyone else to date...
[16:05] <cjwatson> amitk: how did the full build go?
[16:06] <amitk> cjwatson: still building. :-/ I don't have the fastest machine...
[16:06]  * NCommander waits for his build to complete
[16:07] <rtg> amitk: what package are you building?
[16:09] <calc> so the linux-libc-dev headers issue will be fixed tomorrow?
[16:10] <amitk> rtg: the kernel
[16:10]  * calc thinks it may be the cause of some of his OOo build failures atm
[16:10] <amitk> calc: we are looking at it
[16:10] <calc> ok
[16:11] <amitk> rtg: local builds seem to do the right thing (fakeroot debian/rules binary-arch-headers), I am trying a full build.
[16:12] <rtg> amitk: do the right thing with respect to what? I unpacked linux-libc-dev_2.6.30-3.4_amd64.deb and indeed there is nothing in the asm directory. Looks like errno.h got moved to asm-generic
[16:14] <cjwatson> rtg: userspace expects <asm/errno.h> to exist
[16:14] <rtg> cjwatson: so, this worked until I published linux-meta?
[16:14] <cjwatson> no, linux-libc-dev 2.6.30-3.4 inexplicably broke it
[16:15] <cjwatson> -2.3 had <asm/*.h> as one might expect
[16:15] <cjwatson> BTW errno.h was always in asm-generic as well (or at least for ages)
[16:15] <cjwatson> so not a move, it's just that asm/ vanished inexplicably
[16:15] <rtg> cjwatson: how is that so? There were no build rule changes between 2.3 and 3.4, nor were there any core linux updates.
[16:16] <cjwatson> that's what we'd all like to know too
[16:16] <cjwatson> nevertheless, it is true :-(
[16:16] <amitk> rtg: right thing meaning it has the .h files under include/asm
[16:16] <rtg> hmm, I'll rebuild -2.3 from scratch. should only take a few minutes
[16:17] <cjwatson> there's quite a bit of real stuff under asm/ BTW - errno.h is a bad example because it's just a shim
[16:18] <cjwatson> rtg: the build log is very strange - it says "INSTALL include/asm (52 files)" but then the verbose output from cpio -pvd says that there's nothing in include/asm/
[16:19] <amitk> right, and if you do it locally, it all works.
[16:19]  * cjwatson tries diffing the build logs
[16:20] <rtg> amitk: I don't get that. my asm directory is empty
[16:20] <cjwatson> ooh, you can reproduce it
[16:20] <cjwatson> don't suppose you could try with V=1 added to hmake?
[16:20] <amitk> rtg: with 'fdr binary-arch-headers'?
[16:21] <rtg> cjwatson: I think something changed from Linus tree that I missed. I'm pretty sure there was a rebase somewhere between 2.3 and 3.4.
[16:21] <cjwatson> rtg: I'm not entirely sure I believe that this is a kernel change
[16:22] <cjwatson> I'm wondering if it's collateral damage from something else
[16:22] <cjwatson> rtg: did you reproduce it with -2.3 or with -3.4?
[16:23] <amitk> rtg: the move of all arch headers from include/asm-<arch> to arch/<arch>/include/asm has been going on since 2.6.28. I believe it is now complete. But 'make headers_install' basically shields us from the underlying changes. (Or does it?)
[16:24] <cjwatson> the build log diff shows newer versions of coreutils, findutils, login and passwd, gcc-4.4 and friends, libselinux1, libsepol1, libvolume-id1, and cpio, as well as a bunch of other build-deps further up the tree
[16:24] <rtg> amitk: using 3.4, the asm directory _does_ have all of the headers when built locally using 'debian/rules binary-arch-headers'. But I just unpacked the result of 'debuild -b' and saw that it did _not_ have stuff in the asm directory. lemme rebuild from scratch just to confirm.
[16:25] <cjwatson> you'll need linux-libc-dev <= 2.6.30-2.3 installed in order to rebuild successfully obviously :-)
[16:26] <rtg> cjwatson: not if I'm building the kernel 'cause it produces linux-libc-dev
[16:26] <cjwatson> the kernel build builds some host binaries as part of its build process, no?
[16:27] <rtg> cjwatson: uh, yeah. seems kind of circular.
[16:27] <NCommander> cjwatson, indeed :-/
[16:27] <cjwatson> NCommander said it failed at scripts/basic/fixdep
[16:27] <cjwatson> so downgrade, then test
[16:27] <rtg> and in fact my build is borked after updating my chroot.
[16:27] <NCommander> cjwatson, any idea how we're going to unbreak the archive?
[16:28] <cjwatson> NCommander: infinity, or in his absence another sysadmin, can do manual builds)
[16:28] <cjwatson> we've done this in the past
[16:28] <cjwatson> this is not the first time that the process of bootstrapping a new release has left the entire world unbuildable :-)
[16:30] <cjwatson> no obvious sign of trouble in the build log diff
[16:30] <NCommander> cjwatson, indeed, although this is the first time I've ever seen a kernel upload break so oddly
[16:32] <cjwatson> this is doing well at being truly weird, but nothing much really surprises me any more :-)
[16:32] <NCommander> cjwatson, well, I dumped a buildd chroot into sbuild, so if its something funny with pkgbinmangler or something else there, this will shake it out
[16:32] <NCommander> I just wish I had a faster machine
[16:33] <rtg> cjwatson: I downgraded to linux-libc-dev_2.6.28-12.43 and things are at least building now. it'll be done in 10 minutes or so.
[16:34] <cjwatson> NCommander: nothing's impossible, but I'd be truly astonished if any of the pkg* things were capable of affecting this. They just don't hook in at the relevant levels
[16:34] <NCommander> rtg, you can build a kernel in 10 minutes?
[16:34] <rtg> NCommander: one arch
[16:34] <NCommander> rtg, I envy your build rig
[16:34] <NCommander> cjwatson, well, I'm not sure what else it could be. We built the same package with ports overlay, and only broke powerpc
[16:35] <cjwatson> NCommander: it's obviously something weird, but I bet you 10 euros (collectable at UDS) that it isn't pkg-create-dbgsym or pkgbinarymangler
[16:35] <NCommander> I'll take that bet :-)
[16:36] <NCommander> cjwatson, thanks, you just reminded me I need to turn some USD into Euros
[16:36] <NCommander> (and to set aside 10 euros for when I loose this bet)
[16:36] <rtg> I thought it was clear this issue is cause by linux-libc-dev_2.6.30-2.3 ? am I confused ?
[16:37] <cjwatson> rtg: how? :-)
[16:37] <rtg> cjwatson: empty asm directory?
[16:37] <cjwatson> linux-libc-dev_2.6.30-3.4 is the one with the empty asm directory
[16:37] <cjwatson> but the question is *why* it has an empty asm directory
[16:38] <cjwatson> the evidence from the build log suggests that it is trying to install those files, but getting lost somewhere along the way
[16:38] <rtg> ah, then I was a little confused
[16:39] <cjwatson> since when people build the same source package locally, it seems to come out with /usr/include/asm/ populated as normal
[16:39] <NCommander> cjwatson, as a counter example, I just built it in pbuilder in a fresh chroot, I got the /usr/include/asm folder normally
[16:39] <NCommander> Oh
[16:39] <NCommander> :-)
[16:40] <cjwatson>   perl /home/cjwatson/src/ubuntu/linux/ubuntu/scripts/headers_install.pl /home/cjwatson/src/ubuntu/linux/ubuntu/arch/x86/include/asm /home/cjwatson/src/ubuntu/linux/ubuntu/debian/tmp-headers/install/include/asm x86 a.out.h auxvec.h boot.h bootparam.h byteorder.h debugreg.h e820.h errno.h fcntl.h ioctl.h ioctls.h ipcbuf.h ist.h kvm.h ldt.h mce.h mman.h msgbuf.h msr-index.h msr.h mtrr.h param.h poll.h posix_types.h ...
[16:41] <cjwatson> ... posix_types_32.h posix_types_64.h prctl.h processor-flags.h ptrace-abi.h ptrace.h resource.h sembuf.h setup.h shmbuf.h sigcontext.h sigcontext32.h siginfo.h signal.h socket.h sockios.h stat.h statfs.h swab.h termbits.h termios.h types.h ucontext.h unistd.h unistd_32.h unistd_64.h vm86.h vsyscall.h; perl /home/cjwatson/src/ubuntu/linux/ubuntu/scripts/headers_install.pl ...
[16:41] <cjwatson> ... /home/cjwatson/src/ubuntu/linux/ubuntu/debian/tmp-headers/arch/x86/include/asm /home/cjwatson/src/ubuntu/linux/ubuntu/debian/tmp-headers/install/include/asm x86 ; touch /home/cjwatson/src/ubuntu/linux/ubuntu/debian/tmp-headers/install/include/asm/.install
[16:41] <cjwatson> is the relevant install command - which is clearly getting run at least with the correct number of arguments
[16:41] <cjwatson> ("clearly" due to "52 files" in the build log)
[16:45] <amitk> local build finished successfully with include/asm populated
[16:45] <cjwatson> NCommander: where did the broken powerpc build happen?
[16:45] <NCommander> cjwatson, on the buildds. Same issue. The same package built in jaunty had no issue
[16:45] <cjwatson> this is almost seeming like a transient bug in find or cpio
[16:45] <cjwatson> anyway, dinner
[16:45] <rtg> cjwatson: well, we just found one in 'sort' a couple of days ago
[16:48] <rtg> amitk: I just synced my chroot 4 hours ago and my local build exhibits the problem.
[16:49] <rtg> damn, I wish I'd have made a full log of the build. guess I'll have to do it again.
[17:00] <amitk> i haven't synced by chroot yet, I will do so after dinner
[17:19] <rtg> amitk: I'm thinking this a perl problem. 'perl /home/rtg/kern/karmic/kern-64/ubuntu-karmic/scripts/headers_install.pl' appears to be doing the right thing, and that script hasn't been touched since Dec 2008.
[17:26] <NCommander> rtg, that script takes an input directory and copies it somewhere else removing compiler.h definitions
[17:26] <NCommander> rtg, I don't think it was affected by headers moving around :-/
[17:27] <NCommander> (nor does it explain why we only get funny failures on the buildds)
[17:27] <rtg> NCommander: I'm getting failures locally.
[17:27] <NCommander> rtg, you are?
[17:27] <NCommander> How?
[17:27] <rtg> sync your mirror
[17:28] <NCommander> rtg, you mean update my chroot?
[17:28] <rtg> yes
[17:28] <NCommander> rtg, I get the normal build failure because of the bad linux-libc-dev, but so far, I can build a good package
[17:29] <rtg> downgrade linux-libc-dev to -2.3, then try to build the install-arch-headers target for the kernel
[17:33] <rtg> NCommander: hmm, actually the issue is with 'find'
[17:34] <NCommander> rtg, ?
[17:37] <NCommander> find was merged May 4th
[17:38] <NCommander> 2.6.30-3.4 was uploaded on the 1st
[17:38] <rtg> NCommander: I updated the bug report
[17:38] <NCommander> linux-ports 2.6.30-1.1 was uploaded yesterday; same issue
[17:40] <rtg> NCommander: so the timing makes sense
[17:40] <NCommander> rtg, er, the kernel upload was before find was updated
[17:40] <NCommander> Very, very odd
[17:40] <NCommander> and TheMuso reported he built in a karmic chroot and reproduced the issue
[17:40] <NCommander> But this is the best candiate I've seen thus far.
[17:41] <NCommander> cjwatson, I owe you 10 dollars.
[17:41] <NCommander> s/dollars/euros
[18:02] <rtg> NCommander: the upload date for linux-libc-dev_2.6.30-2.3 in LP doesn't make sense. I created the Ubuntu-2.6.30-2.3 log entry on April 30, and I'm positive I uploaded the same day.
[18:02] <calc> don't forget the archive was frozen up until some point
[18:03] <rtg> which would explain by 2.3 built correctly and 3.4 did not (because it was after findutils was synced)
[18:03] <calc> ah that was apr 28 when it opened
[18:03] <NCommander> rtg, but we have kernel builds on ia64 and hppa which happened after the sync
[18:03] <NCommander> (I suspect the find bug is our issue, but its damn odd)
[18:04] <rtg> NCommander: dunno, perhaps there was race with findutils
[18:04] <NCommander> rtg, checked the upload dates, didn't seem so
[18:04] <rtg> was a*
[18:04] <rtg> NCommander: well, at any rate, I can absolutely reproduce the problem by hand.
[18:04] <calc> make sure you check build times of eg findutils vs the kernels since there were buildd backlogs too
[18:04] <NCommander> rtg, I'm still waiting for my kernel build to finish
[18:05] <NCommander> calc, I check the time of upload fromt he buildds
[18:05] <calc> iirc the buildd queues were fairly large at one point
[18:05] <calc> ok
[18:05] <NCommander> calc, find is Priority: required, so it will go fairly fast
[18:05] <NCommander> er
[18:05] <NCommander> Or is it important
[18:06] <calc> required
[18:08] <calc> do packages on buildds automatically get updated that are eg required?
[18:09] <calc> or only if something build-dep against a specific version (iow could some of the buildds still had the older working findutils)
[18:10] <calc> ^ that wrt the comment NCommander made about hppa/ia64
[18:10] <NCommander> calc, it just has to be in main
[18:10] <rtg> NCommander: if I downgrade to findutils_4.4.0-2ubuntu4 then all works as expected.
[18:10] <rtg> seems fairly conclusive.
[18:11] <calc> NCommander: so ubuntu buildds automatically do an upgrade daily (or something like that?)
[18:11] <NCommander> calc, I think dist-upgrade is run as part of sbuild, but thats a voodo that I know not
[18:11] <NCommander> rtg, indeed, now we need to rundown what broke in find
[18:11] <rtg> calc: IIRC the xen instances are created from scratch for every package
[18:11]  * NCommander wonders how you can break find :-/
[18:11] <NCommander> rtg, xen only done for PPAs
[18:11] <calc> rtg: oh
[18:11] <rtg> NCommander: and buildds
[18:12] <NCommander> rtg, Xen doesn't exist for powerpc, armel, ia64, hppa :-/
[18:12] <NCommander> That's why devirtualized PPAs are reserved for Canonical employees only
[18:12]  * NCommander would love xen for any of those archs so then we could get PPAs for them
[18:12] <rtg> but xen is used for x86, which is all _I_ care about most days.
[18:16] <calc> armel is the only really useful arch without xen support ;-)
[18:16] <calc> the rest are dead or dying
[18:17] <cjwatson> NCommander: result
[18:17] <calc> so anyone know what is broken about find now?
[18:17] <cjwatson> though still need a patch; let me try diving into findutils if nobody else already is
[18:17] <calc> oh ok
[18:18] <NCommander> cjwatson, still building, but I think I owe you money now
[18:18] <cjwatson> rtg: what was the sort bug you found?
[18:19] <cjwatson> this is interesting, though; the findutils bug does not manifest for me
[18:19] <rtg> cjwatson: yeah, I've been poking at it but can't figure out why it hates the asm directory
[18:19] <cjwatson> I wonder if it has something to do with path length or similar
[18:20] <rtg> Steve L. fixed the sort bug in coreutils.
[18:20] <cjwatson> the changelog is quite short
[18:20] <NCommander> cjwatson, a path length bug is unlikely; linux is longer than asm
[18:20]  * NCommander wonders/hopes upstream is git
[18:21] <cjwatson> I didn't say it was necessarily longer paths that caused the bug :-)
[18:21] <NCommander> Its a git repo
[18:21] <NCommander> We could just bisect
[18:21]  * cjwatson remembers a bug in HP-UX somethingorother that manifested when the argument was a multiple of 16 characters long
[18:21] <NCommander> cjwatson, ........
[18:21] <NCommander> cjwatson, thanks, you just caused my brain to ABEND with SIGILL
[18:25] <NCommander> /usr/bin/pkgsanitychecks: inconsistent /CurrentlyBuilding file, Package: value is  (should be linux)
[18:25] <NCommander> :-/
[18:25] <cjwatson> rtg: what architecture is this?
[18:26] <cjwatson> I tried recreating your directory structure precisely, and the asm files show up as one would expect
[18:26] <rtg> cjwatson: amd64
[18:27] <cjwatson> (we know it happens on i386 too, of course)
[18:28] <cjwatson> rtg: could you get an strace -tt -s 1024 of that find command?
[18:28] <rtg> cjwatson: one sec.
[18:31] <cjwatson> there really aren't many relevant changes to find itself, but there's a gnulib update in there too
[18:33] <rtg> cjwatson: chinstrap.canonical.com:~rtg/log.txt
[18:33] <cjwatson> NCommander: if it's a bug in imported gnulib files, git bisect won't help, since the findutils maintainers don't commit gnulib files to git
[18:34] <NCommander> cjwatson, *grumble :-/* This will be fun.
[18:34] <cjwatson> rtg: thanks, working on that
[18:37] <cjwatson> hmm
[18:38] <cjwatson> I have a hypothesis, which MAY BE WRONG
[18:38] <rtg> isn't that why its a hypothesis?
[18:39] <cjwatson> I'm betting that the newer code in gnulib is attempting to rely on features (specifically some of the *at functions) not present, or perhaps not reliable, in the kernel running on the buildds
[18:39] <cjwatson> rtg: what kernel version is running on your test system?
[18:39] <rtg> 2.6.30-3-server
[18:39] <cjwatson> drat.
[18:39] <NCommander> so much for that hypothesis
[18:40] <cjwatson> not clearly disproven yet, it could just be that Tim is running into yet a different problem from the buildds ;-)
[18:40] <rtg> the buildds are typically 2.6.24, right?
[18:40] <cjwatson> I forget, something like that
[18:40] <cjwatson> rtg: also, what filesystem?
[18:41] <rtg> ext4
[18:41] <NCommander> cjwatson, powerpc is a dapper (.18 kernel), the rest are hardy
[18:42] <cjwatson> dapper was .15, but whatever ...
[18:42] <NCommander> oh, man, my memory going
[18:43] <cjwatson> .8 .10 .12 .15 .17 .20 .22 .24 .27 .28 .30. IIRC.
[18:44]  * NCommander envies cjwatson's memory
[18:44] <alex_joni> NCommander: he probably remembers the hash
[18:44] <rtg> cjwatson: don't get too comfortable with .30, that may yet change.
[18:45] <NCommander> alex_joni, that would be scary
[18:45] <cjwatson> rtg: yeah :)
[18:45] <cjwatson> NCommander: nah, just if you work with something for six months at a time it tends to settle in long-term memory
[18:45] <NCommander> cjwatson, my kernel build is finishing, I'll have another datapoint for you on this
[18:46]  * NCommander is trying to determine what powerpc, armel, amd64, i386 and lpia have in common that ia64 and hppa don't.
[18:54] <NCommander> bbiab, lunch
[18:55] <cjwatson> sorry, I have a fractious baby here and I think I'll need to feed her before attempting to dig into findutils any more
[19:19] <cjwatson> rtg: could you try this to get a bit more information out of find?  strace -tt -s 1024 find -D tree,search,stat,rates,opt,exec ...
[19:19] <cjwatson> (I'm just hitting it with a bigger hammer in the hope of extracting something useful about its logic)
[19:23] <rtg> cjwatson: one sec
[19:27] <rtg> cjwatson: chinstrap.canonical.com:~rtg/log2.txt
[19:33] <cjwatson> rtg: thanks. argh. no visible difference between asm (breaks) and drm (works).
[19:33] <cjwatson> rtg: I don't suppose I could get an account on this system?
[19:33] <cjwatson> I'm going to need to gdb this
[19:34] <rtg> cjwatson: gimme a bit, its pretty well buried behind a firewall.
[19:43] <cjwatson> phew, baby is asleep, touch wood
[19:54] <cjwatson> rtg: ok, now I'm *really* confused
[19:55] <rtg> cjwatson: dchroot -c karmic-amd64 ?
[19:55] <cjwatson> cjwatson@tyler-b:/home/rtg/kern/karmic/kern-64/ubuntu-karmic/debian/tmp-headers/install/include$ find . -name . -o -name .\* -prune -o -print | grep asm/
[19:55] <cjwatson> ./asm/bootparam.h
[19:55] <cjwatson> oh, whoops :-)
[19:55] <rtg> its a Jaunty box by default
[19:56] <cjwatson> I have no user in the chroot, but maybe that doesn't matter
[19:56] <rtg> cjwatson: do it as root, though I think chroots are bind mounted to /home
[19:57] <cjwatson> ok
[19:57] <cjwatson> reproduced, anyway - going to build myself a test findutils
[19:59] <cjwatson> installing gdb in the chroot
[20:09] <cjwatson> aha, the bug is governed by the presence of the file ..install.cmd
[20:09] <cjwatson> now to determine *why*, although the form of that name permits a good guess
[20:12] <rtg> cjwatson: kind of a strange file name. I din't think it was even legal
[20:13] <cjwatson> file names can contain anything except \0
[20:14] <cjwatson> (yes, even \n, though expect a lot of breakage if you do that!)
[20:14] <rtg> cjwatson: are you thinking the '..' is suspicious?
[20:14] <cjwatson> yes, I'm thinking that find is misinterpreting it as a change of directory level
[20:14] <cjwatson> find is pretty twisty though so it's going to take me a little while to find out exactly where
[20:14] <rtg> cjwatson: why just that directory? there are a bunch of those file names.
[20:15] <rtg> though we might not even notice if those other directories didn't get copied
[20:15] <cjwatson> maybe it needs to be at the top level or something? I'm not sure yet
[20:16] <cjwatson> again, just a hypothesis so far
[20:16] <cjwatson> but I can reproduce the bug on my own system if I touch include/..install.cmd
[20:39] <cjwatson> So. What's happening is that ..install.cmd matches .* and therefore triggers the -prune action. Now, normally -prune is a no-op for files. Unfortunately, in this case the stat buffer find is using is undefined (because it uses FTS_NOSTAT as an optimisation) and so happens to contain stat information from the previous thing it touched, which happened to be a directory; it therefore gets confused and thinks ..install.cmd ...
[20:39] <cjwatson> ... is a directory so sets the "stop_at_current_level" flag. This doesn't get cleared until the next time it exits a directory on its walk, which happens to be when it exits asm.
[20:41] <cjwatson> I'll take this up with upstream, but should be able to construct a band-aid
[20:41] <rtg> cjwatson: I have to wonder why anyone is messing with find to begin with. its so dang old that you'd think all of the bugs had been wrung out of  it. why fix what ain't broken?
[20:43] <cjwatson> 'cos all software still has bugs :-)
[20:43] <rtg> cjwatson: are you done with my server? I normally power it off when not in use as it is kinda noisy.
[20:43] <cjwatson> yeah, thanks
[20:43] <rtg> np
[20:44] <cjwatson> I'll have a go at constructing a minimal test case for findutils upstream
[20:44] <cjwatson> one of the bits of software I work on in my spare time (ho ho) is man-db. man is also ancient. When I took it on it was a buggy pile of shite ...
[21:37] <cjwatson> I think this will fix it but will obviously test:
[21:38] <cjwatson> http://paste.ubuntu.com/166326/
[21:55] <cjwatson> rtg: can I suggest that we should reopen 373214 against the kernel? even though it isn't actually a bug in the kernel, the bug won't in practice be fixed until the kernel is reuploaded
[22:05] <NCommander> cjwatson, I managed to reproduce it, I can give you a shell on my laptop
[22:18] <cjwatson> NCommander: no need, I've got it all now :)
[22:20] <NCommander> cjwatson, +1, and +10 euros to you at UDS
[22:23] <cjwatson> rtg,NCommander: can you guys upload new linux/linux-ports, please, so that it can build against the new findutils when lamont is ready? it might fail to build first time round but don't worry about that
[22:23] <cjwatson> s/might/will/ I guess
[22:23] <NCommander> cjwatson, I need a sponsor
[22:24] <cjwatson> rtg: can you sponsor NCommander? I need to crash ...
[22:24]  * TheMuso is around
[22:24] <NCommander> TheMuso, oh, cool. I need a SPARC patch to go in to fix that FTBFS, but I need to go poof until about 00-01 UTC
[22:25] <TheMuso> NCommander: right
[22:29] <lamont> NCommander: (et al) if you do the upload, pretty please make it build-depend the new findutils
[22:30] <lamont> I don't care if it fails to build, but I'll make you upload again if it succeeds and doesn't have include/asm :-)
[22:30] <cjwatson> err, I think it's just a Build-Conflicts
[22:30] <cjwatson> Build-Conflicts: findutils (= 4.4.1-1ubuntu1)
[22:39] <cjwatson> for anyone who's interested, http://launchpadlibrarian.net/26447089/findutils_4.4.1-1ubuntu1_4.4.1-1ubuntu2.diff.gz is the basic fix; a more complete patch with a test case is on its way upstream


[00:48] <thumper> axw: hey there
[00:49] <thumper> axw: sinzui_ is having issues with your hp fix branch, seems to have a bad import
[00:52] <axw> thumper: reading the email now
[00:53] <axw> eh wtf
[00:54] <axw> aw crap, I think I enabled goimports and it mucked about with the imports
[00:55] <thumper> heh
[00:55] <thumper> axw: so, after talking with fwereade this morning, I'm going to implement a system ssh key right now
[00:55] <axw> thumper: excellent
[00:56]  * axw feels vindicated
[00:56] <axw> ;)
[00:56] <thumper> davecheney: I don't suppose your ssh go stuff has keygen?
[00:59] <axw> thumper: AFAIK they
[00:59] <axw> are just PEM keys
[01:00] <thumper> axw: I'm not entirely sure what that means
[01:01] <axw> thumper: same format used by the private keys we generate already
[01:01] <axw> for mongo
[01:01] <thumper> oh?
[01:01] <thumper> hmm...
[01:02] <axw> thumper: in fact, we could perhaps just use the same one
[01:02] <thumper> axw: maybe...
[01:02] <axw> not sure what the requirements are for SSH
[01:02] <thumper> yeah, well luckily I have another machine here I can ssh to to test
[01:03] <davecheney> thumper: it wouldn't be hard to write
[01:04] <thumper> davecheney: yeah, axw said as much
[01:04] <thumper> I think what I'd like is a function in utils/ssh called "GenerateKey" (surprise)
[01:04] <thumper> that generates to byte arrays,
[01:05] <thumper> one private key and one public key
[01:05] <thumper> equivalents of "id_rsa" and "id_rsa.pub"
[01:05] <thumper> given the core go team's affinity for crypto
[01:05] <thumper> chances are all I need is glue
[01:06] <davecheney> yup
[01:06] <davecheney> lemmie look in the ssh package tests
[01:06] <davecheney> we probably have some code
[01:07] <thumper> davecheney: awesome sauce
[01:07]  * thumper channels fwereade
[01:31] <thumper> davecheney: found any? I've found some docs as to the file format, so could hack something up
[01:31] <thumper> davecheney: but you sould probably be saving me time
[01:36] <davecheney> thumper: yup
[01:36] <davecheney> looking after lunch
[01:43] <thumper> ok
[01:43] <thumper> ta
[01:47] <axw> thumper: I'm just looking to see if we can reuse the existing private key we have
[01:47] <axw> there's code in go.crypto/ssh to generate an authorized_keys line from an RSA public key
[01:55] <axw> thumper: https://gist.github.com/axw/8016123
[01:56] <thumper> axw: what about the identity file that you can hand off to ssh?
[01:57] <axw> thumper: I just fed in the key we give to mongo, added the output of that program to ~/.ssh/authorized_keys, and used ssh -i local-private-key.pem
[01:57] <axw> thumper: it's just the key file
[01:57] <thumper> axw: and that works?
[01:57] <axw> yup
[01:57] <thumper> awesome!
[01:58] <thumper> axw: let me poke around in the code and see if I can get this working nicely...
[01:59] <thumper> axw: I think perhaps we should default to use the system key for 'juju ssh'
[01:59] <thumper> axw: that would make the windows code work nicer
[01:59] <thumper> axw: you are doing the ssh util work for windows yes?
[02:01] <axw> thumper: I'm not so sure we should be passing that key around in the .jenv though, should we?
[02:01] <axw> that would make it difficult to revoke access
[02:01] <thumper> axw: oh, interesting point
[02:01] <axw> thumper: I will be working on making it work for windows, yes
[02:02] <thumper> hmm...
[02:02] <thumper> axw: so, what are your thoughts on that?
[02:03] <thumper> we want a system identity
[02:03] <thumper> and if we can use the same that we have for mongo, then that should be fine
[02:03] <thumper> but we still need something to watch bootstrap with
[02:03] <thumper> perhaps that is a different id?
[02:03] <axw> thumper: sure, I just don't think we should carry it around outside the system; ideally we should dispose of it after bootstrap
[02:04] <axw> from the client I mean
[02:04] <thumper> we want something that will work out of the box with windows users
[02:04] <thumper> axw: but I think we should create a default user key...
[02:04] <thumper> or...
[02:05] <thumper> perhaps we just create the default user key if there is no local one found
[02:05] <thumper> and that is a separate issue
[02:05] <thumper> from the bootstrap issue
[02:05] <axw> thumper: yeah I see your point. I guess we do want another one
[02:05] <axw> we want the system one to be different to the default one used by the CLI
[02:05] <thumper> ok, my branch will work to make this system identiy
[02:05] <thumper> work
[02:05] <axw> because of revocation
[02:06] <axw> sounds good
[02:06] <thumper> axw: yes agree to that last statement
[02:07] <axw> thumper: I think bootstrap can just create another key if it doesn't have one
[02:07] <axw> for the CLI
[02:07]  * thumper nods
[02:07] <thumper> axw: that is a separate bit of work though
[02:07] <axw> thumper: yes, I can do that later
[02:07] <thumper> lets go with the system identity, and using that for the bootstrap
[02:07] <thumper> ...
[02:15] <axw> thumper: what about CLI users other than the bootstrapper?
[02:16] <thumper> not sure yet
[02:16] <thumper> I'm actually trying to find the code where it loads the authorized keys
[02:16] <thumper> the code I'm reading seems to indicate that it doesn't bother
[02:16] <thumper> unless I set them explicitly
[02:16] <thumper> but I know this is wrong...
[02:18] <thumper> ugh
[02:18] <thumper> an OR
[02:19] <axw> thumper: maybe just updated environs.BootstrapContext to add the key to whatever's computed?
[02:30] <thumper> axw: is the ca-private-key the thing we use to talk to mongo?
[02:30] <thumper> axw: if so, that is in the .jenv file in the bootstrap config
[02:30] <thumper> so perhaps we do want a different identity?
[02:30] <thumper> something that gets generated and written to cloud init, but not on the client side
[02:30] <axw> thumper: it is in there, but I think fwereade wants it out
[02:31] <axw> thumper: and yes that's what we use for mongo
[02:31] <thumper> ah, fair call
[02:31]  * thumper goes to make coffee
[02:31] <thumper> this takes more thinking than I thought :)
[02:31] <thumper> the identity stuff that is
[02:31] <thumper> not the making coffee
[02:31] <axw> hehe
[02:32] <axw> not enough coffee makes making coffee more difficult
[03:39]  * axw -> lunch
[03:48] <hazmat> wrt to put charm api, it appears to be unconditionally setting the scheme to local, deploy cli landing should resolve
[04:28] <thumper> axw: are you good with me updating our go.crypto to tip?
[04:29] <thumper> also, how do I find the full hash of an hg tip revision?
[04:51] <thumper> axw: in the end, I have gone with a ssh.GenerateKey function
[04:51] <thumper> axw: I'm going to poke things tomorrow and test
[04:51] <thumper> but for today, I'm done
[04:51] <thumper> night all
[07:33] <axw> jam: I *think* the streams.canonical.com URL it's going to is correct, but what's there at the moment is in the wrong place
[07:34] <jam> axw: that's sort of what I expect as well, and we don't really want to be using it anyway
[07:34] <axw> that's the impression I got from wallyworld
[07:34] <jam> the bigger question is what else did we try (and fail) and why
[07:34] <axw> yep
[07:34] <jam> and they don't want to use --debug because of security purposes, but that's where we dump all the URLs we try
[07:35] <axw> It'd be good to see that in "verbose output" (that never got landed did it?)
[07:35] <axw> thumper's stuff
[07:36] <jam> verbose was never intended to dump step-by-step stuff,(I think) I think it was meant to stay in --debug, we just dump other useful but-maybe-dangerous stuff there.
[07:36] <axw> when I'm bootstrapping without --debug, it can sit there doing image/tools stuff for a fair while without any feedback
[07:36] <jam> --verbose is more about user info about what we're doing
[07:37] <axw> yeah I suppose that is a bit too much info for general consumption
[07:38] <jam> I think we also have a bug that we don't write the debug information for the one that actually worked :)
[07:39] <axw> heh
[07:40] <rogpeppe1> mornin' all
[07:40] <axw> hey rogpeppe1
[07:41] <rogpeppe1> axw: sorry i didn't manage to do your review yesterday - am looking at it now
[07:41] <axw> rogpeppe1: nps, thanks. actually it gave me time to make it a bit nicer :)
[07:41] <rogpeppe1> axw: my first thought when i looked at it was that it could probably be factored out into a separate little type
[08:06] <dimitern> rogpeppe1, jam, https://codereview.appspot.com/38540044/ if you can please
[08:12] <rogpeppe1> dimitern: will do
[08:12] <dimitern> ta!
[08:17] <axw> rogpeppe1: do we need go.crypto pinned at the rev it's at?
[08:17] <axw> rogpeppe1: thumper's doing some SSH key stuff and was wondering if we can move to the tip
[08:18] <rogpeppe1> axw: not as far as i'm aware
[08:18] <rogpeppe1> axw: it would be good to try tip
[08:18] <axw> ok
[08:18] <rogpeppe1> axw: but it may not be possible
[08:18] <rogpeppe1> axw: it might have some go1.2 deps
[08:18] <axw> yeah ok, will have to check that
[08:18] <axw> thanks
[08:18] <jam> axw: so... do you know why we need a juju-run process to be running if we are just ssh'ing into each machine anyway?
[08:19] <axw> jam: I was under the impression it was not long running
[08:19]  * axw re-reads thumper's email
[08:19] <jam> axw: tim's statement was that it couldn't be the existing code because the existing stuff wasn't long lived
[08:19] <jam> at least, I thought I remembered seeing that
[08:20] <axw> jam: are you referrng to this statement? `"juju run" is really syntactic sugar around "juju ssh" and the server
[08:20] <axw> component "juju-run" that does the execution inside a hook context.`
[08:21] <jam> axw: so I know Tim has a Runner object that sits on a port and lets hooks fire queries into it (the way the current one does, but apparently somehow different)
[08:22] <jam> it might be that we "juju ssh" into each machine, fire up that juju-run process, and then have it (or something else) spawn of clients that run the script you wrote.
[08:22] <jam> I'm not sure how it actually all ties together.
[08:22] <jam> it sounded ilke you ssh in to poke the juju-run process
[08:22] <jam> but it might be that you are doing "juju ssh juju-run myscript.bash"
[08:22] <jam> well
[08:23] <jam> for i in ???; do juju ssh $i juju ssh /usr/bin/juju-run myscript.bash; done
[08:23] <rogpeppe1> jam: are there some design docs for juju-run somewhere?
[08:23] <axw> jam: sorry, I haven't actually looked at the code yet - not entirely sure. that's not how I thought it worked. what I thought was that you ssh'd in, then ran "juju-run <hook> <command>", and <command> was executed within the hook context
[08:24]  * axw looks at code
[08:25] <axw> jam: so, I have only taken a very brief look, but it looks to me that the runner/socket is just for that process
[08:25] <axw> it's just reusing the existing jujuc code
[08:26] <axw> hrm, or maybe not
[08:29] <jam> axw: related code, but definitely a different object
[08:30] <jam> can anyone look at: https://codereview.appspot.com/40670043/ its been sitting for a week now
[08:31] <axw> jam: I see; in that case, I'm not sure why SSH is required here.
[08:31] <axw> there's still the machine case, but that's a bit different anyway
[08:31] <axw> jam: I'll take a look
[08:34] <axw> jam: re the juju-run stuff: how else would you do it? you'd need to put something in state, right? and then record the results there?
[08:42] <dimitern> rogpeppe1, is it good to land?
[08:42] <rogpeppe1> dimitern: sorry, still reviewing axw's
[08:43] <dimitern> rogpeppe1, ok
[08:53] <jam> axw: so it depends what you want as a result. If you're meant to just trigger a script to run, yes. If you're meant to end up in an interactive session on the remote machine, then you'd have to ssh
[08:54] <jam> axw: an open question, is this intended to run sequentially on all machines, or would all machines fire up at the same time
[08:54] <jam> if parallel, then interactive session (or even printing to the screen) is pretty useless
[08:54] <jam> if sequential, then this really scales poorly when you have 100s of machines
[08:55] <axw> jam: my understanding is it's meant to be parallel and non-interactive, but I'm not 100% sure
[08:55] <axw> fwereade: hah, good point about the nonces
[08:56] <axw> fwereade: yes, pretty certain it was always "manual:..."
[08:56] <axw> I will double check before basing anything on that tho
[08:56] <fwereade> axw, cool, thanks
[08:56] <fwereade> axw, I'm just on destroy-environment at last, sorry delay
[08:57] <axw> fwereade: cool
[08:58] <jam> axw: if non-interactive, then it seems like you *don't* want to ssh into each one. (you don't really want to fire off 100 ssh connections all at once, either)
[09:00] <jam> btw, hi fwereade
[09:00] <fwereade> jam, heyhey
[09:01] <rogpeppe1> axw: you have a review
[09:01] <axw> rogpeppe1: thanks
[09:02] <fwereade> jam, axw: `juju run` is non-interactive, there's juju ssh for direct interactivity
[09:02] <axw> right
[09:03] <fwereade> jam, axw: the actual degree of parallelism required has not been specified, but a number of ideas have been floated around the concept
[09:03] <jam> fwereade: so some review questions... Do we want to have 'remove-machine' in a 1.16 branch? It has been approved, but it is more of a do-we-want-this-policy
[09:04] <fwereade> jam, axw: round-robin has been mentioned; parallel has been mentioned; parallel with horrid hacked-on interactivity has ever come up
[09:05] <fwereade> jam, what do we gain from a new 1.16 with that in?
[09:05] <jam> fwereade: or it could be driven from the db without ssh at all?
[09:05] <jam> fwereade: compatibility, especially with docs
[09:05] <jam> though you still have "you must have newer than X"
[09:06] <jam> but it would make "remove-machine" available in a stable release
[09:07] <axw> jam: the only thing I see being an issue with going through the db is that there's no ephemeral node support to handle the edge cases around client termination
[09:07] <axw> jam: it's not interactive, but there's still a session to maintain
[09:09] <axw> rogpeppe1: "This should probably allow itself to be stopped when closed.j"  -- do you mean use an additional select with a time.After?
[09:09] <rogpeppe1> axw: yeah
[09:09] <axw> thanks
[09:10] <jam> axw: I don't quite see how it is different from how we fire hooks normally, it is just a custom script instead of a charm hook
[09:10] <rogpeppe1> dimitern: reviewed
[09:10] <dimitern> rogpeppe1, thank
[09:10] <dimitern> s
[09:10] <axw> jam: the main difference is that the result needs to live somewhere. who owns that?
[09:12] <fwereade> jam, sorry, I missed the context on remove-machine
[09:13] <fwereade> jam, did you have any thoughts re the more human vocabulary I handwaved around in that thread yesterday?
[09:13] <jam> axw: seems easy enough to put it into the DB as a result of running the request
[09:14] <fwereade> jam, axw: questions of how much we store and for how long become interesting
[09:14] <jam> fwereade: so, there was a discussion that we should use "remove-*" for stuff instead of "destroy-*" and it is easy to add those aliases into a 1.16 release. I have a fairly trivial patch up, just waiting on whether we *want* to do it. Its approved, but I wanted to run it by you.
[09:14] <fwereade> jam, so, indeed, I think I am now up to speed on it
[09:14] <jam> fwereade: as for your natural language stuff, I think you had interesting points, but part of that is what we can evolve into, and we can have a consistent lower level terminology
[09:15] <jam> fwereade: for juju-run, if you aggregate the results somewhere (like the DB) then you do need a way to see it (is the suggestion that normally it just prints to your console via ssh?) that still seems like a whole lot of spam unless you very carefully craft the script to give enough context so you can figure out which machine actually "failed"
[09:16] <jam> so either, there isn't much result, so it doesn't matter, or there is some result, but you'd really rather have that logged and aggregated
[09:16] <jam> you *can* do "juju run foo.sh >aggregate.this.txt"
[09:42] <rogpeppe1> jam, fwereade: is there a design doc for juju-run? i'd like to contribute to the discussion, but i don't actually know what the goals are.
[10:23] <axw> rogpeppe1: updated
[10:26] <rogpeppe1> axw: re: "shouldn't this use the mutex?"
[10:26] <rogpeppe1> axw: it looks to me as if inst.InstanceId is inside the embedded ec2.Instance
[10:26] <rogpeppe1> axw: and hence should be protected by the same mutex
[10:26] <axw> rogpeppe1: gah, so it is
[10:27] <axw> rogpeppe1: sorry, will fix it now
[10:27] <rogpeppe1> axw: thanks
[10:29] <jam> rogpeppe1: I don't know of one, but tim is the one driving it
[10:38] <axw> rogpeppe1: updated again, sorry about that
[10:38] <rogpeppe1> axw: np
[10:41] <axw> bbl
[10:45] <jam> rogpeppe1: axw, mgz_: standup
[10:45] <jam> https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig
[12:38] <jam> fwereade: poke about where to put worker/provisioner/authentication.go
[12:41] <fwereade> jam, how do you feel about environs?
[12:42] <jam> fwereade: so "NewAPIAuthenticator" doesn't feel like it should be in environs
[12:42] <jam> and NewEnvironAuthenticator feels ok there
[12:43] <jam> the issue is that they both want to share the actual underlying implementation
[12:43] <fwereade> jam, well, it's about giving access to a <handwave> environ
[12:43] <jam> which just has a way to get a StateInfo and then populate it with its final machine password stuff
[12:43] <fwereade> jam, I'm at least halfway inclined to just give it its own package somewhere
[12:44] <fwereade> jam, the only thing I'm sure of is that it's not exclusive to worker/provisioner, but it is something to do with setting up instances
[12:44] <jam> fwereade: arguably the simpleAuth code should be exposed in state/ and the others lay on top of that?
[12:44] <fwereade> jam, state doesn't seem very correct, in every case but bootstrap I think it's api-ey really
[12:47] <jam> well, the api server uses it internally
[12:47] <jam> which is why NewEnviron* is still necessary
[12:57] <fwereade> natefinch, https://codereview.appspot.com/36290043/patch/190001/200015 is going to collide with you, isn't it?
[13:05] <natefinch> fwereade, ish.   I sometimes see Error: Test left sockets in a dirty state but no other errors (although that happens on trunk as well - has done for a while)
[13:19] <mattyw> ^^ any idea what might be going wrong?
[13:19] <jam> mattyw: we've had some issues with flaky tests, I just wanted to know if this was caused by your change. As it doesn't seem related, I'll just submit again.
[13:20] <jam> fwereade: when are we running mongo without ssl?
[13:20] <mattyw> jam, ok thanks - I'll keep an eye on it
[14:01] <natefinch> man, this USB ethernet adapter crashing my laptop is not cool (especially given there is no ethernet plug on the laptop).....
[14:02] <natefinch> anyway, I have to go snowblow my driveway, which will probably  take me a couple hours all told.
[14:03] <natefinch> jam, rogpeppe1: I verified that the test failure I'm having definitely is only happening with mongo 2.2, and not 2.4.  Not sure exactly why yet, but now that I can A/B test easily, I should be able to figure it out.
[14:06] <fwereade> jam, sorry I missed you -- it's for the store... but I am suddenly suspicious of the Right Thing to do there
[14:06] <fwereade> jam, I think it's probably better to factor out the existing store test harness and use that
[14:07] <fwereade> jam, than to introduce a fresh entanglement between the two
[14:20] <rogpeppe1> wow, my internet is super flaky today
[14:20] <rogpeppe1> can anyone see this?
[14:28] <mattyw> rogpeppe1, I can
[14:29] <rogpeppe1> mattyw: ok, that's something at least :-)
[14:29] <mattyw> rogpeppe1, also - I'm getting a "x509: certificate is valid for *" error when trying to connect to the core api on 17070 (it seems to happend in my call to websocket.Dial)
[14:30] <mattyw> is there a particular version of websocket I should be using do you think?
[14:30] <rogpeppe1> mattyw: could you paste the code that you're using to dial?
[14:30] <rogpeppe1> mattyw: in general, you should be using the state/api package to connect to the API
[14:30] <rogpeppe1> mattyw: (assuming you're using Go)
[14:31] <mattyw> rogpeppe1, I am, I was just kinda messing around with connecting to the websocket in go
[14:32] <rogpeppe1> mattyw: you can connect if you use InsecureSkipVerify
[14:32] <mattyw> rogpeppe1, 	ws, err := websocket.Dial("wss:/adr:17070", "", "http://localhost")
[14:33] <mattyw> ah, I'm not doing any tls at all
[14:33] <rogpeppe1> mattyw: otherwise your client won't know about the root CA cert that's used for the server
[14:33] <rogpeppe1> mattyw: you *are* doing tls
[14:33] <rogpeppe1> mattyw: that's the "wss:" thing
[14:33] <mattyw> rogpeppe1, but I'm not setting up any tls config
[14:34] <rogpeppe1> mattyw: ah yes.
[14:34] <mattyw> (which is what I meant in my head at least :))
[14:34] <rogpeppe1> mattyw: look at how it's done in state/api/apiclient.go:/Open
[14:34] <mattyw> rogpeppe1, will do, thanks again
[14:35] <axw> rogpeppe1: https://codereview.appspot.com/43650045 exercises the Addresses/Refresh code in provider/ec2
[14:35] <rogpeppe1> axw: looking
[14:35] <axw> rogpeppe1: I updated provider/ec2 again, there was a bug
[14:35] <mattyw> rogpeppe1, thanks for the acme-friendly shortcut ;)
[14:36] <rogpeppe1> mattyw: are you using acme now?
[14:38] <rogpeppe1> axw: what was the bug?
[14:40] <axw> rogpeppe1: Addresses was holding the lock around refresh(), which called inst.Id() and took the lock
[14:40] <axw> well, tried to
[14:41] <axw> so, deadlock
[14:41] <rogpeppe1> axw: ah, good thing i suggested the test then :-)
[14:41] <axw> very
[14:43] <rogpeppe1> axw: i still think we should use AddScripts rather than AddFile - someone might change AddFile to use the cloudinit write_files feature, which would then potentially break the nonce.txt logic
[14:43] <axw> rogpeppe1: fair point
[14:43] <axw> ok, I'll change it
[14:43] <rogpeppe1> axw: thanks
[14:43]  * rogpeppe1 goes for lunch
[14:55] <jamespage> hazmat, re mongodb build without scripting - which scons option are you referring to?  I can toggle between v8 and spidermonkey in our current source package (2.4.8) but I can't figure out how to disable it completely
[14:55] <axw> rogpeppe1: can you please approve the ec2test MP when you're back? I'm not in the group
[14:56] <hazmat> jamespage, usev8
[14:56] <axw> rogpeppe1: and/or merge it if there's no bot
[14:56] <hazmat> jamespage, https://github.com/mongodb/mongo/blob/master/SConstruct#L452
[14:57] <hazmat> jamespage, it comes up a few times.. agreed its not clear if its  toggle for spidermonkey or disable
[14:57] <jamespage> hazmat, it certainly was
[14:57] <jamespage> but upstream don't ship spidermonkey any longer
[14:57] <jamespage> "--usesm"
[14:59] <hazmat> jamespage, so reaching out to upstream sounds like the thing to do.. i don't see the usesm option in trunk scons.
[14:59] <hazmat> either that or trying a build without the v8 option
[14:59] <jamespage> hazmat, no its gone
[14:59] <jamespage> hazmat, if you don't specify it default to v8
[14:59] <jamespage> I'll poke at it a bit
[14:59]  * hazmat fires up a build on a spare server
[15:30] <dimitern> jam, rogpeppe1, https://codereview.appspot.com/43860043 AddCharm via the API
[15:35] <axw> fwereade: the nonce for manual bootstrap nodes does not start with "manual:". do you think it's still okay to use it, documenting that?
[15:36] <rogpeppe1> niemeyer: does this look reasonable to you? https://codereview.appspot.com/43650045/
[15:37] <axw> fwereade: that case *could* be detected by checking that id == "0" && extracting the provider type from EnvironConfig, but that's pretty heavy handed
[15:37] <fwereade> axw, hmm, possibly, I don't suppose that one has any useful distinguishing features?
[15:37] <axw> fwereade: not that I've thought of so far
[15:37] <niemeyer> rogpeppe1: What happens in practice?
[15:37] <niemeyer> rogpeppe1: I mean, what is the real EC2 doing about this?
[15:37] <rogpeppe1> niemeyer: the DNS name appears some time after the instance has been started
[15:38] <axw> fwereade: its instance-id is "manual:", but we've established that's not good enough to rely on
[15:38] <rogpeppe1> niemeyer: so any realistic client will need to poll it
[15:38] <niemeyer> rogpeppe1: Sounds good then
[15:38] <rogpeppe1> niemeyer: cool, thanks
[15:38] <niemeyer> rogpeppe1: np, thanks for asking
[15:39] <rogpeppe1> axw: will submit now
[15:39] <fwereade> axw, you know, I'm not so bothered by the weight as I am by the correctness
[15:39] <fwereade> axw, it's not a cost we'll be paying often enough to worry about, I think
[15:39] <axw> rogpeppe1: thanks
[15:40] <axw> fwereade: true
[15:40] <axw> fwereade: in fact, I won't even be passing it in in my case, because I don't check manager machines
[15:40] <fwereade> axw, thought that might be the case
[15:41] <axw> fwereade: so, now might be a good time to change the provider name from "null" to "manual"
[15:41] <fwereade> axw, please still test that behaviour in case we do ever end up with demoted machine 0s ;)
[15:41] <axw> certainly
[15:41] <fwereade> axw, most certainly, but I suspect that may take some finessing around upgrades
[15:42] <fwereade> axw, *maybe* you could get away with registering the same provider twice under different names..?
[15:43] <axw> hmm maybe, will have to see. I can make this code check for either "null" or "manual" for now
[15:47] <rogpeppe1> axw: FYI i just got this error when bootstrapping:
[15:47] <rogpeppe1> Connection to ec2-54-196-224-38.compute-1.amazonaws.com closed.
[15:47] <rogpeppe1> error: cannot open state: cannot create log collection: unauthorized mongo access: unauthorized
[15:47] <rogpeppe1> axw: i haven't looked into it but i suspect something is trying to talk to mongo before it's ready
[15:49] <axw> rogpeppe1: hmm, there's no changes regarding mongo communications in synchronous bootstrap
[15:50] <axw> rogpeppe1: are you on trunk?
[15:50] <rogpeppe1> axw: yeah
[15:51] <axw> rogpeppe1: with my just-merged branch?
[15:51] <axw> I'll test it again
[15:51] <rogpeppe1> axw: hmm, not sure
[15:51] <axw> rogpeppe1: shouldn't actually matter, just curious
[15:51] <rogpeppe1> axw: it's the kind of thing that wouldn't happen very often, if it's what i suspect
[15:52] <rogpeppe1> axw: i'll see if it happens again
[15:53] <rogpeppe1> axw: your goamz branch is now merged
[15:53] <axw> rogpeppe1: great, thanks
[15:54] <axw> rogpeppe1: ah, I should've updated dependencies.tsv... will do that now
[15:54] <rogpeppe1> axw: good point
[15:54] <rogpeppe1> axw: revno 44
[15:54] <axw> thanks
[15:56] <rogpeppe1> axw: ah, it's not bootstrap's fault at all
[15:56] <rogpeppe1> axw: i just wasn't printing enough log messages
[15:57] <axw> rogpeppe1: thanks for checking
[15:59]  * rogpeppe1 reboots
[16:10] <axw> night all
[16:14] <dimitern> natefinch, ping
[16:14] <dimitern> natefinch, can you take a look at this please? since roger is not around. https://codereview.appspot.com/43860043
[16:22] <natefinch> dimitern, sure
[16:47] <mattyw> fwereade, I guess we should have a chat about the next stage of the juju id stuff
[17:14] <natefinch> dimitern, can we put Info in the charm.Repository interface? I'd much prefer that than doing some casting to anonymous interfaces to access that method.
[17:19] <dimitern> natefinch, actually, I wasn't sure about using Info() or maybe just lazy not to calculate the sha256 from the already downloaded file
[17:20] <dimitern> natefinch, i'll change that to use only Get and read and calculate the hash from the downloaded charm
[17:21] <natefinch> that also works
[18:36] <rogpeppe> dimitern: you've got a review
[18:36] <rogpeppe> natefinch, dimitern: i made an alternative suggestion
[18:36] <rogpeppe> natefinch, dimitern: which hopefully works out a bit simpler than calculating the hash yourself
[19:01]  * rogpeppe is done for the day
[19:04] <rogpeppe> darn juju-restore is destroying its own re-bootstrapped machine
[19:04] <rogpeppe> g'night all
[19:29] <natefinch> thumper, morning
[19:29] <thumper> o/ natefinch
[19:29]  * thumper trawls through the overnight emails
[20:26] <natefinch> thumper, how's your knowledge of mongo?
[20:27] <thumper> natefinch: virtually non-existent
[20:27] <natefinch> thumper, heh.  My HA tests are failing on 2.2 but passing on 2.4.... and I can't figure out why.
[20:40] <thumper> hmm... trying to bootstrap ec2 failed with an error around simple streams
[20:40] <thumper> WARNING no tools available, attempting to retrieve from https://streams.canonical.com/juju
[20:40] <thumper> ERROR bootstrap failed: cannot find bootstrap tools: XML syntax error on line 9: element <hr> closed by </body>
[20:56] <natefinch> thumper sounds like you might be getting a 404 not found page instead of the XML you're supposed to get, and it's trying to parse the page's HTML
[20:56] <thumper> natefinch: that's what I thought too
[20:56] <thumper> but since I actually needed to upload tools
[20:56] <thumper> it was a convenient error
[20:57] <natefinch> heh
[20:57] <natefinch> thumper, it does worry me that we're not checking the http error code before trying to parse what's returned, though
[20:58] <thumper> heh
[20:58] <thumper> yeah
[20:59] <thumper> should file a bug
[20:59] <thumper> I'm now confused...
[20:59] <thumper> with my work
[21:01] <thumper> WTF!
[21:01] <thumper> natefinch: hangout?
[21:01] <thumper> it might help if I talk this through
[21:03] <natefinch> sure... one sec
[21:05] <thumper> natefinch: I worked it out :)
[21:05] <natefinch> thumper, ha, ok, cool
[21:05] <thumper> dumb permissions...
[21:06] <natefinch> thumper, heh, something like that happened when I was building mongo from source yesterday.  30 minute build, fails at the very end with a permissions issue.
[21:06] <thumper> I have code that now creates a juju system ssh identity file and adds to the authorized keys
[21:06] <thumper> and it works for ec2 and local
[21:07] <thumper> so probably everything
[21:07] <thumper> although care should be taken with manual
[22:42] <thumper> wallyworld_: o/
[22:43] <wallyworld_> heloooo
[22:43] <thumper> wallyworld_: good break?
[22:43] <wallyworld_> so many emails
[22:43] <thumper> :)
[22:43] <wallyworld_> yeah, no internet
[22:43] <wallyworld_> great result in the cricket :-D
[22:43] <thumper> wallyworld_: I'm heading off to the gym shortly, but we have a scheduled call in just over 2 hours
[22:43] <wallyworld_> yep
[22:43] <thumper> so we can chat then
[22:43] <wallyworld_> ok


[15:45] <mup> Bug #2017147 opened: spurious dns entries for dhdcp leases in interface set with same mac address; entry won't cleanup when lease expired <MAAS:New> <https://launchpad.net/bugs/2017147>
[16:15] <mup> Bug #2017155 opened: "Different channel detected" error with an air-gapped deployment <MAAS:New> <https://launchpad.net/bugs/2017155>
[19:43] <mup> Bug #2016908 opened: udev fails to make prctl() syscall with apparmor=0 (as used by maas by default) <MAAS:New> <maas-images:Invalid> <linux (Ubuntu):Triaged> <systemd (Ubuntu):Invalid> <https://launchpad.net/bugs/2016908>
[19:43] <mup> Bug #2017168 opened: Columns in "Controllers" tab disappear depending on window size <MAAS:New> <https://launchpad.net/bugs/2017168>
[19:55] <mup> Bug #2016908 changed: udev fails to make prctl() syscall with apparmor=0 (as used by maas by default) <MAAS:New> <maas-images:Invalid> <linux (Ubuntu):Triaged> <systemd (Ubuntu):Invalid> <https://launchpad.net/bugs/2016908>
[19:55] <mup> Bug #2017168 changed: Columns in "Controllers" tab disappear depending on window size <MAAS:New> <maas-ui:New> <https://launchpad.net/bugs/2017168>
[20:01] <mup> Bug #2016908 opened: udev fails to make prctl() syscall with apparmor=0 (as used by maas by default) <MAAS:New> <maas-images:Invalid> <linux (Ubuntu):Triaged> <systemd (Ubuntu):Invalid> <https://launchpad.net/bugs/2016908>
[20:01] <mup> Bug #2017168 opened: Columns in "Controllers" tab disappear depending on window size <MAAS:New> <https://launchpad.net/bugs/2017168>